2024-04-02T11:15:17.390 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T11:15:17.391 DEBUG:teuthology.run:Teuthology command: teuthology --name matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --verbose --archive /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494 --owner scheduled_matan@teuthology -- /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/orig.config.yaml 2024-04-02T11:15:17.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T11:15:17.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494 branch: wip-matanb-crimson-testing-apr-2 description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7635494' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-testing-apr-2 ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 38caedd896c6b19ed5e43655372d95dc8e78cfda ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 38caedd896c6b19ed5e43655372d95dc8e78cfda selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 107e1624f2a9b72a56ee529969558944d24a2a75 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 223 sha1: 38caedd896c6b19ed5e43655372d95dc8e78cfda sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 107e1624f2a9b72a56ee529969558944d24a2a75 targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJK+nFThnXLgUX2cmMcAQA4Ue5WcKpH70qrYfnnQQBrUA/tKnCFvCD3dWiMXtv168DqA/XIP9BUTn5gZh8XTUc= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBILXb1sCtBCsjkTFXqvIZz4kqBeJ8pjSF1jv1Hq7ql4vvYJOoCxP9hUF/7C9k5YUBVahFiqqeq05+Sz6GbqJOyU= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:03:04 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T11:15:17.496 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa; will attempt to use it 2024-04-02T11:15:17.496 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa/tasks 2024-04-02T11:15:17.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T11:15:17.498 INFO:teuthology.task.internal:Checking packages... 2024-04-02T11:15:17.519 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '38caedd896c6b19ed5e43655372d95dc8e78cfda' 2024-04-02T11:15:17.519 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T11:15:17.519 INFO:teuthology.packaging:ref: None 2024-04-02T11:15:17.519 INFO:teuthology.packaging:tag: None 2024-04-02T11:15:17.519 INFO:teuthology.packaging:branch: wip-matanb-crimson-testing-apr-2 2024-04-02T11:15:17.519 INFO:teuthology.packaging:sha1: 38caedd896c6b19ed5e43655372d95dc8e78cfda 2024-04-02T11:15:17.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-testing-apr-2 2024-04-02T11:15:17.660 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2589.g38caedd8 2024-04-02T11:15:17.662 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T11:15:17.667 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T11:15:17.667 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T11:15:17.675 INFO:teuthology.task.internal:Saving configuration 2024-04-02T11:15:17.687 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T11:15:17.693 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T11:15:17.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 11:10:15.409715', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJK+nFThnXLgUX2cmMcAQA4Ue5WcKpH70qrYfnnQQBrUA/tKnCFvCD3dWiMXtv168DqA/XIP9BUTn5gZh8XTUc='} 2024-04-02T11:15:17.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 11:10:15.408007', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBILXb1sCtBCsjkTFXqvIZz4kqBeJ8pjSF1jv1Hq7ql4vvYJOoCxP9hUF/7C9k5YUBVahFiqqeq05+Sz6GbqJOyU='} 2024-04-02T11:15:17.737 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T11:15:17.743 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-04-02T11:15:17.743 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-02T11:15:17.743 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T11:15:17.813 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe8831bdf70>, signals=[15]) 2024-04-02T11:15:17.814 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T11:15:17.820 INFO:teuthology.task.internal:Opening connections... 2024-04-02T11:15:17.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T11:15:17.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:15:17.909 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T11:15:17.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:15:17.998 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T11:15:18.004 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-04-02T11:15:18.021 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-04-02T11:15:18.021 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-04-02T11:15:18.077 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T11:15:18.078 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T11:15:18.078 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-04-02T11:15:18.100 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-02T11:15:18.116 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-02T11:15:18.116 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-02T11:15:18.176 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T11:15:18.177 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-02T11:15:18.177 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T11:15:18.177 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T11:15:18.177 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T11:15:18.177 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-02T11:15:18.200 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T11:15:18.208 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T11:15:18.213 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T11:15:18.213 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-04-02T11:15:18.215 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-02T11:15:18.234 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T11:15:18.240 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T11:15:18.240 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-04-02T11:15:18.271 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-02T11:15:18.405 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T11:15:18.462 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T11:15:18.476 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-02T11:15:18.476 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-02T11:15:18.476 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T11:15:18.477 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T11:15:18.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T11:15:18.492 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-02T11:15:18.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T11:15:18.493 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-04-02T11:15:18.550 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-361.el9.x86_64 2024-04-02T11:15:18.550 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.14.0-361.el9.x86_64 2024-04-02T11:15:18.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum install -y kernel 2024-04-02T11:15:18.552 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-361.el9.x86_64 2024-04-02T11:15:18.552 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-361.el9.x86_64 2024-04-02T11:15:18.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-04-02T11:15:20.665 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-02T11:15:20.939 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-02T11:15:22.124 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-04-02T11:15:22.540 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-04-02T11:15:23.439 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-02T11:15:23.706 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-02T11:15:25.281 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-02T11:15:29.124 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.1 MB 00:00 2024-04-02T11:15:31.874 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 9.7 kB/s | 16 kB 00:01 2024-04-02T11:15:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-02T11:15:38.917 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-02T11:15:41.216 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T11:15:41.216 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T11:15:41.314 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T11:15:41.325 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:15:41.326 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 76 M 2024-04-02T11:15:41.327 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 123 M 2024-04-02T11:15:41.327 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:15:41.486 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T11:15:41.994 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-04-02T11:15:42.553 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 27 MB/s | 34 MB 00:01 2024-04-02T11:15:42.870 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 20 MB/s | 28 MB 00:01 2024-04-02T11:15:42.873 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:15:42.873 INFO:teuthology.orchestra.run.smithi138.stdout:Total 49 MB/s | 76 MB 00:01 2024-04-02T11:15:42.874 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:15:43.241 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:15:43.242 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:15:44.083 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:15:44.083 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:15:46.409 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:15:46.994 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T11:15:47.081 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:15:48.263 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 824 kB/s | 19 MB 00:23 2024-04-02T11:15:49.855 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:15:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:15:52.121 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-04-02T11:15:53.530 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-02T11:15:54.758 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:15:54.808 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:15:55.267 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-02T11:15:59.521 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:00.281 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-04-02T11:16:02.546 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T11:16:02.546 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T11:16:02.662 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T11:16:02.672 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:Install 4 Packages 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 76 M 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 123 M 2024-04-02T11:16:02.673 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:16:02.824 INFO:teuthology.orchestra.run.smithi133.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T11:16:03.341 INFO:teuthology.orchestra.run.smithi133.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-04-02T11:16:03.849 INFO:teuthology.orchestra.run.smithi133.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 29 MB/s | 34 MB 00:01 2024-04-02T11:16:04.200 INFO:teuthology.orchestra.run.smithi133.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 20 MB/s | 28 MB 00:01 2024-04-02T11:16:04.203 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:16:04.203 INFO:teuthology.orchestra.run.smithi133.stdout:Total 50 MB/s | 76 MB 00:01 2024-04-02T11:16:04.203 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:16:04.566 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:16:04.566 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:16:05.414 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:16:05.414 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:16:07.702 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:16:08.355 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T11:16:08.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:16:11.163 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:16:11.301 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:16:16.012 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:16:16.099 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:20.835 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:28.999 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:29.000 INFO:teuthology.orchestra.run.smithi138.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T11:16:29.000 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:16:29.042 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:29.391 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:29.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T11:16:29.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:16:29.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:16:29.821 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:16:30.156 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T11:16:30.257 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:30.257 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T11:16:30.258 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T11:16:30.258 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T11:16:30.258 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T11:16:30.258 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-04-02T11:16:30.258 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T11:16:30.258 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T11:16:30.337 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:30.338 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T11:16:30.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-04-02T11:16:30.372 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:16:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T11:16:30.374 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T11:16:30.374 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T11:16:30.374 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-04-02T11:16:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:16:30.374 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T11:16:30.375 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T11:16:30.375 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-434.el9.x86_64 2024-04-02T11:16:30.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T11:16:30.966 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-04-02T11:16:32.349 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T11:16:32.389 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-04-02T11:16:32.392 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-02T11:16:32.406 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.3keEECqcul 2024-04-02T11:16:32.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3keEECqcul 2024-04-02T11:16:32.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.3keEECqcul 2024-04-02T11:16:32.611 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.3keEECqcul is 6KB 2024-04-02T11:16:32.660 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.3keEECqcul 2024-04-02T11:16:32.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T11:16:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T11:16:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T11:16:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T11:16:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T11:16:32.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T11:16:32.875 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-04-02T11:16:50.104 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:50.104 INFO:teuthology.orchestra.run.smithi133.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T11:16:50.104 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:50.147 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:50.519 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:50.519 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T11:16:50.519 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T11:16:50.519 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:50.952 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:16:51.298 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T11:16:51.437 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:51.437 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T11:16:51.438 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T11:16:51.438 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T11:16:51.438 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T11:16:51.438 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-02T11:16:51.438 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T11:16:51.438 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T11:16:51.518 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T11:16:51.518 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T11:16:51.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-04-02T11:16:51.552 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 68.el9 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8264254 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T11:16:51.553 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T11:16:51.554 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T11:16:51.555 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-04-02T11:16:51.555 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:16:51.555 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T11:16:51.555 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T11:16:51.556 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 5.14.0-434.el9.x86_64 2024-04-02T11:16:51.556 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T11:16:52.261 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-04-02T11:16:53.657 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T11:16:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-04-02T11:16:53.691 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-04-02T11:16:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.TBTd4gePSY 2024-04-02T11:16:53.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TBTd4gePSY 2024-04-02T11:16:53.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.TBTd4gePSY 2024-04-02T11:16:53.911 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.TBTd4gePSY is 6KB 2024-04-02T11:16:53.963 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.TBTd4gePSY 2024-04-02T11:16:53.977 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T11:16:54.041 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T11:16:54.041 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T11:16:54.041 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T11:16:54.042 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T11:16:54.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T11:16:54.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-04-02T11:17:02.888 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-04-02T11:17:02.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-02T11:17:02.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:17:24.220 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-02T11:17:24.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-02T11:17:24.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:18:02.910 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T11:18:11.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-02T11:18:11.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:18:13.381 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-02T11:18:13.859 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-02T11:18:13.859 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-434.el9.x86_64"... 2024-04-02T11:18:13.859 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-02T11:18:13.873 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-434.el9.x86_64 2024-04-02T11:18:13.874 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T11:18:13.874 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T11:18:13.874 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T11:18:14.875 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-02T11:18:14.876 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T11:18:14.956 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-02T11:18:15.009 DEBUG:teuthology.parallel:result is None 2024-04-02T11:18:24.223 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T11:18:33.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-02T11:18:33.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:18:33.772 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-02T11:18:34.238 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-02T11:18:34.238 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-434.el9.x86_64"... 2024-04-02T11:18:34.239 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-04-02T11:18:34.253 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-434.el9.x86_64 2024-04-02T11:18:34.253 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T11:18:34.253 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T11:18:34.253 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T11:18:35.253 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-02T11:18:35.254 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T11:18:35.328 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-04-02T11:18:35.370 DEBUG:teuthology.parallel:result is None 2024-04-02T11:18:35.371 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T11:18:35.378 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T11:18:35.378 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T11:18:35.380 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T11:18:35.399 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T11:18:35.407 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T11:18:35.413 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T11:18:35.413 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T11:18:35.436 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T11:18:35.536 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T11:18:35.543 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T11:18:35.543 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T11:18:35.545 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T11:18:35.590 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T11:18:35.590 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T11:18:35.604 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T11:18:35.604 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T11:18:35.606 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T11:18:35.614 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T11:18:35.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T11:18:35.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T11:18:35.646 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T11:18:35.655 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T11:18:35.655 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T11:18:35.683 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T11:18:35.702 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T11:18:35.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T11:18:35.823 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T11:18:35.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T11:18:35.947 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:18:35.947 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T11:18:36.016 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T11:18:36.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T11:18:36.116 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T11:18:36.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T11:18:36.245 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:18:36.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T11:18:36.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-04-02T11:18:36.316 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-02T11:18:36.362 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T11:18:36.419 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T11:18:36.913 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T11:18:36.922 INFO:teuthology.task.internal:Starting timer... 2024-04-02T11:18:36.922 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T11:18:36.938 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T11:18:36.946 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-02T11:18:36.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-04-02T11:18:37.004 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-04-02T11:18:37.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-02T11:18:37.072 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-02T11:18:37.074 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T11:18:37.074 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-04-02T11:18:37.104 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-04-02T11:18:37.105 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-02T11:18:37.126 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-02T11:18:37.126 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-02T11:18:37.126 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T11:18:37.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T11:18:37.172 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T11:18:37.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T11:18:37.201 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T11:18:37.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-04-02T11:18:37.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-02T11:18:37.285 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T11:18:37.295 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T11:18:37.311 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T11:18:37.312 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T11:22:56.025 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-02T11:22:56.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-02T11:22:56.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:22:56.115 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-02T11:22:56.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-02T11:22:56.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-02T11:22:56.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T11:22:56.284 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-02T11:22:56.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-02T11:22:56.357 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T11:22:56.369 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T11:22:56.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T11:22:56.369 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T11:22:56.372 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T11:22:56.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T11:22:56.400 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T11:22:56.417 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T11:22:56.428 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T11:22:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T11:22:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-04-02T11:22:56.460 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-04-02T11:22:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-02T11:22:56.477 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T11:22:56.488 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-02T11:22:56.493 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T11:22:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T11:22:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T11:22:56.545 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-02T11:22:56.549 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T11:22:56.549 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-02T11:22:56.550 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.550 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.550 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.550 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.571 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-02T11:22:56.576 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T11:22:56.576 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-02T11:22:56.577 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.577 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.577 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.577 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T11:22:56.577 INFO:teuthology.run_tasks:Running task install... 2024-04-02T11:22:56.586 DEBUG:teuthology.task.install:project ceph 2024-04-02T11:22:56.586 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda'}} 2024-04-02T11:22:56.586 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda'} 2024-04-02T11:22:56.586 INFO:teuthology.task.install:Using flavor: crimson 2024-04-02T11:22:56.592 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T11:22:56.592 INFO:teuthology.task.install:extra packages: [] 2024-04-02T11:22:56.592 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda', 'tag': None, 'wait_for_package': False} 2024-04-02T11:22:56.593 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=38caedd896c6b19ed5e43655372d95dc8e78cfda 2024-04-02T11:22:56.595 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda', 'tag': None, 'wait_for_package': False} 2024-04-02T11:22:56.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=38caedd896c6b19ed5e43655372d95dc8e78cfda 2024-04-02T11:22:56.809 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/ 2024-04-02T11:22:56.809 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2589.g38caedd8 2024-04-02T11:22:56.816 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/ 2024-04-02T11:22:56.816 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2589.g38caedd8 2024-04-02T11:22:56.925 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T11:22:56.925 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:22:56.926 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T11:22:56.954 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T11:22:56.955 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38caedd896c6b19ed5e43655372d95dc8e78cfda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T11:22:56.977 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-apr-2/38caedd896c6b19ed5e43655372d95dc8e78cfda/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T11:22:56.978 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:22:56.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T11:22:57.006 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T11:22:57.006 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38caedd896c6b19ed5e43655372d95dc8e78cfda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T11:22:57.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T11:22:57.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T11:22:57.103 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T11:22:57.154 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T11:22:57.169 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-02T11:22:57.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-02T11:22:57.221 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-04-02T11:22:57.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-04-02T11:22:57.417 INFO:teuthology.orchestra.run.smithi138.stdout:74 files removed 2024-04-02T11:22:57.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-04-02T11:22:57.461 INFO:teuthology.orchestra.run.smithi133.stdout:74 files removed 2024-04-02T11:22:57.488 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw 2024-04-02T11:22:58.141 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 218 kB/s | 87 kB 00:00 2024-04-02T11:22:58.186 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 217 kB/s | 87 kB 00:00 2024-04-02T11:22:58.495 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 45 kB/s | 12 kB 00:00 2024-04-02T11:22:58.500 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-02T11:22:58.796 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 7.8 kB/s | 2.0 kB 00:00 2024-04-02T11:22:58.859 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.4 kB/s | 2.0 kB 00:00 2024-04-02T11:22:59.206 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-02T11:22:59.487 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-02T11:23:00.590 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-04-02T11:23:01.004 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-04-02T11:23:01.464 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-02T11:23:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-02T11:23:08.840 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.1 MB 00:00 2024-04-02T11:23:10.349 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-02T11:23:11.301 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 825 kB/s | 8.0 MB 00:09 2024-04-02T11:23:11.820 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-02T11:23:13.227 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-02T11:23:16.817 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-02T11:23:18.436 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-04-02T11:23:19.417 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 9.6 M 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-04-02T11:23:19.429 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.4 M 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.7 M 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.9 M 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 19 M 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 25 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 795 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T11:23:19.430 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 548 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.0 M 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 45 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 134 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 159 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 315 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 294 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 99 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T11:23:19.431 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:Install 30 Packages 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 2 Packages 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:19.432 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 58 M 2024-04-02T11:23:19.433 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:23:20.045 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-02T11:23:21.170 INFO:teuthology.orchestra.run.smithi133.stdout:(1/32): ceph-radosgw-19.0.0-2589.g38caedd8.el9. 6.9 MB/s | 9.6 MB 00:01 2024-04-02T11:23:21.220 INFO:teuthology.orchestra.run.smithi133.stdout:(2/32): ceph-selinux-19.0.0-2589.g38caedd8.el9. 506 kB/s | 25 kB 00:00 2024-04-02T11:23:21.337 INFO:teuthology.orchestra.run.smithi133.stdout:(3/32): libcephfs2-19.0.0-2589.g38caedd8.el9.x8 6.7 MB/s | 795 kB 00:00 2024-04-02T11:23:21.488 INFO:teuthology.orchestra.run.smithi133.stdout:(4/32): ceph-base-19.0.0-2589.g38caedd8.el9.x86 3.5 MB/s | 5.9 MB 00:01 2024-04-02T11:23:21.488 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-02T11:23:21.513 INFO:teuthology.orchestra.run.smithi133.stdout:(5/32): libradosstriper1-19.0.0-2589.g38caedd8. 3.1 MB/s | 548 kB 00:00 2024-04-02T11:23:21.571 INFO:teuthology.orchestra.run.smithi133.stdout:(6/32): python3-ceph-argparse-19.0.0-2589.g38ca 774 kB/s | 45 kB 00:00 2024-04-02T11:23:21.630 INFO:teuthology.orchestra.run.smithi133.stdout:(7/32): python3-ceph-common-19.0.0-2589.g38caed 2.3 MB/s | 134 kB 00:00 2024-04-02T11:23:21.697 INFO:teuthology.orchestra.run.smithi133.stdout:(8/32): python3-cephfs-19.0.0-2589.g38caedd8.el 2.3 MB/s | 159 kB 00:00 2024-04-02T11:23:21.780 INFO:teuthology.orchestra.run.smithi133.stdout:(9/32): python3-rados-19.0.0-2589.g38caedd8.el9 3.7 MB/s | 315 kB 00:00 2024-04-02T11:23:21.964 INFO:teuthology.orchestra.run.smithi133.stdout:(10/32): ceph-common-19.0.0-2589.g38caedd8.el9. 8.7 MB/s | 19 MB 00:02 2024-04-02T11:23:21.997 INFO:teuthology.orchestra.run.smithi133.stdout:(11/32): python3-rbd-19.0.0-2589.g38caedd8.el9. 1.3 MB/s | 294 kB 00:00 2024-04-02T11:23:22.023 INFO:teuthology.orchestra.run.smithi133.stdout:(12/32): python3-rgw-19.0.0-2589.g38caedd8.el9. 1.7 MB/s | 99 kB 00:00 2024-04-02T11:23:22.123 INFO:teuthology.orchestra.run.smithi133.stdout:(13/32): librgw2-19.0.0-2589.g38caedd8.el9.x86_ 7.9 MB/s | 5.0 MB 00:00 2024-04-02T11:23:22.199 INFO:teuthology.orchestra.run.smithi133.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 528 kB/s | 106 kB 00:00 2024-04-02T11:23:22.223 INFO:teuthology.orchestra.run.smithi133.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 593 kB/s | 44 kB 00:00 2024-04-02T11:23:22.274 INFO:teuthology.orchestra.run.smithi133.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 704 kB/s | 159 kB 00:00 2024-04-02T11:23:22.308 INFO:teuthology.orchestra.run.smithi133.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 661 kB 00:00 2024-04-02T11:23:22.332 INFO:teuthology.orchestra.run.smithi133.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2024-04-02T11:23:22.358 INFO:teuthology.orchestra.run.smithi133.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 498 kB/s | 41 kB 00:00 2024-04-02T11:23:22.383 INFO:teuthology.orchestra.run.smithi133.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-02T11:23:22.408 INFO:teuthology.orchestra.run.smithi133.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 930 kB/s | 46 kB 00:00 2024-04-02T11:23:22.433 INFO:teuthology.orchestra.run.smithi133.stdout:(22/32): qatlib-service-24.02.0-1.el9.x86_64.rp 463 kB/s | 35 kB 00:00 2024-04-02T11:23:22.551 INFO:teuthology.orchestra.run.smithi133.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-02T11:23:22.659 INFO:teuthology.orchestra.run.smithi133.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-02T11:23:22.684 INFO:teuthology.orchestra.run.smithi133.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-04-02T11:23:22.751 INFO:teuthology.orchestra.run.smithi133.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-04-02T11:23:22.818 INFO:teuthology.orchestra.run.smithi133.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-04-02T11:23:22.877 INFO:teuthology.orchestra.run.smithi133.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-04-02T11:23:22.902 INFO:teuthology.orchestra.run.smithi133.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T11:23:22.936 INFO:teuthology.orchestra.run.smithi133.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-02T11:23:23.211 INFO:teuthology.orchestra.run.smithi133.stdout:(31/32): librbd1-19.0.0-2589.g38caedd8.el9.x86_ 12 MB/s | 3.7 MB 00:00 2024-04-02T11:23:23.420 INFO:teuthology.orchestra.run.smithi133.stdout:(32/32): librados2-19.0.0-2589.g38caedd8.el9.x8 6.3 MB/s | 3.4 MB 00:00 2024-04-02T11:23:23.423 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:23:23.424 INFO:teuthology.orchestra.run.smithi133.stdout:Total 15 MB/s | 58 MB 00:03 2024-04-02T11:23:23.424 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:23:23.458 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:23:23.458 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:23:23.933 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:23:23.934 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:23:25.376 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:23:25.835 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-02T11:23:25.871 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:25.945 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:26.034 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 3/34 2024-04-02T11:23:26.056 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:26.102 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:26.145 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-02T11:23:26.169 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-02T11:23:26.194 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-02T11:23:26.217 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-02T11:23:26.233 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-02T11:23:26.443 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-02T11:23:26.475 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-02T11:23:26.480 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-02T11:23:26.830 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-02T11:23:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:26.913 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:26.956 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9. 12/34 2024-04-02T11:23:27.011 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 13/34 2024-04-02T11:23:27.049 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 14/34 2024-04-02T11:23:27.138 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-02T11:23:27.165 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 16/34 2024-04-02T11:23:27.254 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 16/34 2024-04-02T11:23:27.289 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-02T11:23:27.400 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-02T11:23:27.440 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-02T11:23:27.480 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-02T11:23:27.514 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-02T11:23:27.542 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-02T11:23:27.574 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-02T11:23:27.613 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-02T11:23:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-02T11:23:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 26/34 2024-04-02T11:23:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 26/34 2024-04-02T11:23:28.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 27/34 2024-04-02T11:23:28.324 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x8 28/34 2024-04-02T11:23:29.149 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 9.6 M 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.4 M 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.7 M 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T11:23:29.159 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.9 M 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 19 M 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 25 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 795 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 548 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.0 M 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T11:23:29.160 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 45 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 134 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 159 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 315 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 294 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 99 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T11:23:29.161 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout:Install 30 Packages 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-04-02T11:23:29.162 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:29.163 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 58 M 2024-04-02T11:23:29.163 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:23:30.631 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:30.670 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:31.089 INFO:teuthology.orchestra.run.smithi138.stdout:(1/32): ceph-common-19.0.0-2589.g38caedd8.el9.x 12 MB/s | 19 MB 00:01 2024-04-02T11:23:31.173 INFO:teuthology.orchestra.run.smithi138.stdout:(2/32): ceph-base-19.0.0-2589.g38caedd8.el9.x86 3.6 MB/s | 5.9 MB 00:01 2024-04-02T11:23:31.198 INFO:teuthology.orchestra.run.smithi138.stdout:(3/32): ceph-selinux-19.0.0-2589.g38caedd8.el9. 232 kB/s | 25 kB 00:00 2024-04-02T11:23:31.265 INFO:teuthology.orchestra.run.smithi138.stdout:(4/32): libradosstriper1-19.0.0-2589.g38caedd8. 8.0 MB/s | 548 kB 00:00 2024-04-02T11:23:31.332 INFO:teuthology.orchestra.run.smithi138.stdout:(5/32): libcephfs2-19.0.0-2589.g38caedd8.el9.x8 4.9 MB/s | 795 kB 00:00 2024-04-02T11:23:31.376 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:31.407 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 30/34 2024-04-02T11:23:31.432 INFO:teuthology.orchestra.run.smithi138.stdout:(6/32): python3-ceph-argparse-19.0.0-2589.g38ca 451 kB/s | 45 kB 00:00 2024-04-02T11:23:31.482 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 30/34 2024-04-02T11:23:31.482 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T11:23:31.482 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T11:23:31.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:31.525 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:31.541 INFO:teuthology.orchestra.run.smithi138.stdout:(7/32): python3-ceph-common-19.0.0-2589.g38caed 1.2 MB/s | 134 kB 00:00 2024-04-02T11:23:31.616 INFO:teuthology.orchestra.run.smithi138.stdout:(8/32): librgw2-19.0.0-2589.g38caedd8.el9.x86_6 14 MB/s | 5.0 MB 00:00 2024-04-02T11:23:31.641 INFO:teuthology.orchestra.run.smithi138.stdout:(9/32): python3-cephfs-19.0.0-2589.g38caedd8.el 1.6 MB/s | 159 kB 00:00 2024-04-02T11:23:31.667 INFO:teuthology.orchestra.run.smithi138.stdout:(10/32): python3-rados-19.0.0-2589.g38caedd8.el 6.2 MB/s | 315 kB 00:00 2024-04-02T11:23:31.784 INFO:teuthology.orchestra.run.smithi138.stdout:(11/32): ceph-radosgw-19.0.0-2589.g38caedd8.el9 4.2 MB/s | 9.6 MB 00:02 2024-04-02T11:23:31.809 INFO:teuthology.orchestra.run.smithi138.stdout:(12/32): python3-rgw-19.0.0-2589.g38caedd8.el9. 700 kB/s | 99 kB 00:00 2024-04-02T11:23:31.859 INFO:teuthology.orchestra.run.smithi138.stdout:(13/32): python3-rbd-19.0.0-2589.g38caedd8.el9. 1.3 MB/s | 294 kB 00:00 2024-04-02T11:23:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-04-02T11:23:31.951 INFO:teuthology.orchestra.run.smithi138.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 44 kB 00:00 2024-04-02T11:23:31.993 INFO:teuthology.orchestra.run.smithi138.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-04-02T11:23:32.052 INFO:teuthology.orchestra.run.smithi138.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 661 kB 00:00 2024-04-02T11:23:32.077 INFO:teuthology.orchestra.run.smithi138.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2024-04-02T11:23:32.105 INFO:teuthology.orchestra.run.smithi138.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 380 kB/s | 41 kB 00:00 2024-04-02T11:23:32.128 INFO:teuthology.orchestra.run.smithi138.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-02T11:23:32.153 INFO:teuthology.orchestra.run.smithi138.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 700 kB/s | 35 kB 00:00 2024-04-02T11:23:32.178 INFO:teuthology.orchestra.run.smithi138.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 618 kB/s | 46 kB 00:00 2024-04-02T11:23:32.303 INFO:teuthology.orchestra.run.smithi138.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-02T11:23:32.403 INFO:teuthology.orchestra.run.smithi138.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-02T11:23:32.429 INFO:teuthology.orchestra.run.smithi138.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-04-02T11:23:32.496 INFO:teuthology.orchestra.run.smithi138.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-04-02T11:23:32.621 INFO:teuthology.orchestra.run.smithi138.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-02T11:23:32.654 INFO:teuthology.orchestra.run.smithi138.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-02T11:23:32.730 INFO:teuthology.orchestra.run.smithi138.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-04-02T11:23:32.822 INFO:teuthology.orchestra.run.smithi138.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-02T11:23:33.047 INFO:teuthology.orchestra.run.smithi138.stdout:(31/32): librbd1-19.0.0-2589.g38caedd8.el9.x86_ 12 MB/s | 3.7 MB 00:00 2024-04-02T11:23:33.139 INFO:teuthology.orchestra.run.smithi138.stdout:(32/32): librados2-19.0.0-2589.g38caedd8.el9.x8 7.0 MB/s | 3.4 MB 00:00 2024-04-02T11:23:33.143 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:23:33.143 INFO:teuthology.orchestra.run.smithi138.stdout:Total 15 MB/s | 58 MB 00:03 2024-04-02T11:23:33.143 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:23:33.178 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:23:33.178 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:23:33.654 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:23:33.655 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:23:35.137 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:23:35.638 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-02T11:23:35.674 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:35.749 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 3/34 2024-04-02T11:23:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:35.913 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/34 2024-04-02T11:23:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/34 2024-04-02T11:23:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/34 2024-04-02T11:23:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-02T11:23:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/34 2024-04-02T11:23:36.181 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-02T11:23:36.211 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/34 2024-04-02T11:23:36.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/34 2024-04-02T11:23:36.657 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:36.732 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:36.791 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9. 12/34 2024-04-02T11:23:36.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 13/34 2024-04-02T11:23:36.880 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 14/34 2024-04-02T11:23:36.969 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 15/34 2024-04-02T11:23:36.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 16/34 2024-04-02T11:23:37.073 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 16/34 2024-04-02T11:23:37.109 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 17/34 2024-04-02T11:23:37.218 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 18/34 2024-04-02T11:23:37.251 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 19/34 2024-04-02T11:23:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 20/34 2024-04-02T11:23:37.316 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 21/34 2024-04-02T11:23:37.345 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 22/34 2024-04-02T11:23:37.376 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 23/34 2024-04-02T11:23:37.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-02T11:23:37.963 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-02T11:23:38.000 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 26/34 2024-04-02T11:23:38.071 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 26/34 2024-04-02T11:23:38.146 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 27/34 2024-04-02T11:23:38.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x8 28/34 2024-04-02T11:23:40.495 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:41.222 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 29/34 2024-04-02T11:23:41.260 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 30/34 2024-04-02T11:23:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 30/34 2024-04-02T11:23:41.352 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T11:23:41.352 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T11:23:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:41.386 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:41.709 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:41.709 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-04-02T11:23:41.710 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:42.686 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 32/34 2024-04-02T11:23:42.710 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 32/34 2024-04-02T11:23:42.711 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:23:42.711 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T11:23:42.711 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T11:23:42.711 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T11:23:42.711 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:42.713 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-02T11:23:42.790 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-02T11:23:42.790 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 1/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 3/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 5/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 6/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 7/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9. 8/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x8 9/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 10/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:43.995 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 12/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 13/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-02T11:23:43.996 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-02T11:23:43.997 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 33/34 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T11:23:45.024 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.025 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.026 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:45.027 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:23:45.384 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-test 2024-04-02T11:23:45.949 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:29 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:23:46.111 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 87 M 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:23:46.112 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 173 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout:Install 11 Packages 2024-04-02T11:23:46.113 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:23:46.114 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 90 M 2024-04-02T11:23:46.114 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 820 M 2024-04-02T11:23:46.114 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:23:47.043 INFO:teuthology.orchestra.run.smithi133.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2024-04-02T11:23:47.176 INFO:teuthology.orchestra.run.smithi133.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-02T11:23:47.227 INFO:teuthology.orchestra.run.smithi133.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-02T11:23:47.268 INFO:teuthology.orchestra.run.smithi133.stdout:(4/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-04-02T11:23:47.319 INFO:teuthology.orchestra.run.smithi133.stdout:(5/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.8 MB/s | 90 kB 00:00 2024-04-02T11:23:47.352 INFO:teuthology.orchestra.run.smithi133.stdout:(6/11): libcephsqlite-19.0.0-2589.g38caedd8.el9 337 kB/s | 173 kB 00:00 2024-04-02T11:23:47.469 INFO:teuthology.orchestra.run.smithi133.stdout:(7/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 3.9 MB/s | 469 kB 00:00 2024-04-02T11:23:47.536 INFO:teuthology.orchestra.run.smithi133.stdout:(8/11): libubsan-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-04-02T11:23:47.737 INFO:teuthology.orchestra.run.smithi133.stdout:(9/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 608 kB/s | 122 kB 00:00 2024-04-02T11:23:48.329 INFO:teuthology.orchestra.run.smithi133.stdout:(10/11): jq-1.6-16.el9.x86_64.rpm 185 kB/s | 187 kB 00:01 2024-04-02T11:23:51.585 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-02T11:23:51.586 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:52.036 INFO:teuthology.orchestra.run.smithi133.stdout:(11/11): ceph-test-19.0.0-2589.g38caedd8.el9.x8 17 MB/s | 87 MB 00:05 2024-04-02T11:23:52.040 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:23:52.040 INFO:teuthology.orchestra.run.smithi133.stdout:Total 15 MB/s | 90 MB 00:05 2024-04-02T11:23:52.082 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:23:52.106 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:23:52.106 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:23:52.606 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 32/34 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 32/34 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T11:23:52.631 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:52.633 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-02T11:23:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-02T11:23:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:52.741 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:23:52.742 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:23:53.774 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:23:53.848 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-04-02T11:23:53.876 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-04-02T11:23:53.896 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 1/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 3/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 4/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 5/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_6 6/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 7/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9. 8/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x8 9/34 2024-04-02T11:23:53.948 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 10/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 11/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 12/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 13/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-02T11:23:53.949 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-02T11:23:53.950 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-02T11:23:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 31/34 2024-04-02T11:23:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-02T11:23:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 33/34 2024-04-02T11:23:53.966 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-02T11:23:53.996 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-04-02T11:23:54.040 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-04-02T11:23:54.085 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-04-02T11:23:54.116 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-04-02T11:23:54.156 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-04-02T11:23:54.199 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-04-02T11:23:54.226 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 10/11 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.121 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T11:23:55.122 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T11:23:55.123 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T11:23:55.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:55.124 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:23:55.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-04-02T11:23:56.035 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:30 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:23:56.192 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 87 M 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: libasan8 x86_64 13.2.1-6.3.el9 appstream 469 k 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 173 k 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-04-02T11:23:56.193 INFO:teuthology.orchestra.run.smithi138.stdout: lksctp-tools x86_64 1.0.19-2.el9 CentOS-BaseOS 90 k 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout:Install 11 Packages 2024-04-02T11:23:56.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:23:56.195 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 90 M 2024-04-02T11:23:56.195 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 820 M 2024-04-02T11:23:56.195 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:23:57.318 INFO:teuthology.orchestra.run.smithi138.stdout:(1/11): oniguruma-6.9.6-1.el9.5.x86_64.rpm 704 kB/s | 217 kB 00:00 2024-04-02T11:23:57.484 INFO:teuthology.orchestra.run.smithi138.stdout:(2/11): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-02T11:23:57.552 INFO:teuthology.orchestra.run.smithi138.stdout:(3/11): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-04-02T11:23:57.577 INFO:teuthology.orchestra.run.smithi138.stdout:(4/11): libcephsqlite-19.0.0-2589.g38caedd8.el9 305 kB/s | 173 kB 00:00 2024-04-02T11:23:57.610 INFO:teuthology.orchestra.run.smithi138.stdout:(5/11): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-04-02T11:23:57.636 INFO:teuthology.orchestra.run.smithi138.stdout:(6/11): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.5 MB/s | 90 kB 00:00 2024-04-02T11:23:57.945 INFO:teuthology.orchestra.run.smithi138.stdout:(7/11): jq-1.6-16.el9.x86_64.rpm 561 kB/s | 187 kB 00:00 2024-04-02T11:23:58.412 INFO:teuthology.orchestra.run.smithi138.stdout:(8/11): libasan8-13.2.1-6.3.el9.x86_64.rpm 604 kB/s | 469 kB 00:00 2024-04-02T11:23:58.579 INFO:teuthology.orchestra.run.smithi138.stdout:(9/11): libubsan-11.4.1-3.el9.x86_64.rpm 295 kB/s | 187 kB 00:00 2024-04-02T11:23:58.846 INFO:teuthology.orchestra.run.smithi138.stdout:(10/11): yaml-cpp-0.6.3-5.el9.x86_64.rpm 281 kB/s | 122 kB 00:00 2024-04-02T11:24:05.863 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 10/11 2024-04-02T11:24:05.933 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 11/11 2024-04-02T11:24:11.987 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 11/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 1/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 2/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-04-02T11:24:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-04-02T11:24:12.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-16.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T11:24:12.424 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T11:24:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T11:24:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-02T11:24:12.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:12.425 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:12.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph 2024-04-02T11:24:13.291 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:57 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:13.444 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 6.5 k 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:24:13.447 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 2.2 M 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 1.6 M 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 249 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.0 M 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 71 M 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T11:24:13.448 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T11:24:13.449 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T11:24:13.450 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:Install 36 Packages 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 84 M 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 922 M 2024-04-02T11:24:13.451 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:24:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:(1/36): ceph-19.0.0-2589.g38caedd8.el9.x86_64.r 15 kB/s | 6.5 kB 00:00 2024-04-02T11:24:14.959 INFO:teuthology.orchestra.run.smithi133.stdout:(2/36): ceph-mgr-19.0.0-2589.g38caedd8.el9.x86_ 1.2 MB/s | 1.6 MB 00:01 2024-04-02T11:24:16.050 INFO:teuthology.orchestra.run.smithi133.stdout:(3/36): ceph-mds-19.0.0-2589.g38caedd8.el9.x86_ 954 kB/s | 2.2 MB 00:02 2024-04-02T11:24:16.267 INFO:teuthology.orchestra.run.smithi133.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2589.g38ca 1.1 MB/s | 249 kB 00:00 2024-04-02T11:24:16.551 INFO:teuthology.orchestra.run.smithi133.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 659 kB/s | 187 kB 00:00 2024-04-02T11:24:16.626 INFO:teuthology.orchestra.run.smithi133.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T11:24:16.676 INFO:teuthology.orchestra.run.smithi133.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 634 kB/s | 32 kB 00:00 2024-04-02T11:24:16.743 INFO:teuthology.orchestra.run.smithi133.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 710 kB/s | 47 kB 00:00 2024-04-02T11:24:16.793 INFO:teuthology.orchestra.run.smithi133.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 777 kB/s | 39 kB 00:00 2024-04-02T11:24:16.885 INFO:teuthology.orchestra.run.smithi133.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-04-02T11:24:17.044 INFO:teuthology.orchestra.run.smithi133.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 7.2 MB/s | 1.1 MB 00:00 2024-04-02T11:24:17.102 INFO:teuthology.orchestra.run.smithi133.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-04-02T11:24:17.161 INFO:teuthology.orchestra.run.smithi133.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-04-02T11:24:17.236 INFO:teuthology.orchestra.run.smithi133.stdout:(14/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.4 MB/s | 187 kB 00:00 2024-04-02T11:24:17.603 INFO:teuthology.orchestra.run.smithi133.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 341 kB/s | 125 kB 00:00 2024-04-02T11:24:17.770 INFO:teuthology.orchestra.run.smithi133.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-02T11:24:17.896 INFO:teuthology.orchestra.run.smithi133.stdout:(17/36): ceph-mon-19.0.0-2589.g38caedd8.el9.x86 1.3 MB/s | 5.0 MB 00:03 2024-04-02T11:24:18.046 INFO:teuthology.orchestra.run.smithi133.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 549 kB/s | 151 kB 00:00 2024-04-02T11:24:18.079 INFO:teuthology.orchestra.run.smithi133.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 237 kB/s | 43 kB 00:00 2024-04-02T11:24:18.163 INFO:teuthology.orchestra.run.smithi133.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-02T11:24:18.764 INFO:teuthology.orchestra.run.smithi133.stdout:(21/36): ceph-osd-19.0.0-2589.g38caedd8.el9.x86 19 MB/s | 71 MB 00:03 2024-04-02T11:24:18.780 INFO:teuthology.orchestra.run.smithi133.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 17 kB/s | 11 kB 00:00 2024-04-02T11:24:18.814 INFO:teuthology.orchestra.run.smithi133.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 354 kB/s | 18 kB 00:00 2024-04-02T11:24:18.856 INFO:teuthology.orchestra.run.smithi133.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 462 kB/s | 358 kB 00:00 2024-04-02T11:24:18.881 INFO:teuthology.orchestra.run.smithi133.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 291 kB/s | 19 kB 00:00 2024-04-02T11:24:18.906 INFO:teuthology.orchestra.run.smithi133.stdout:(26/36): python3-jaraco-collections-3.0.0-8.el9 184 kB/s | 23 kB 00:00 2024-04-02T11:24:18.934 INFO:teuthology.orchestra.run.smithi133.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-04-02T11:24:18.957 INFO:teuthology.orchestra.run.smithi133.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-04-02T11:24:19.007 INFO:teuthology.orchestra.run.smithi133.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 785 kB/s | 79 kB 00:00 2024-04-02T11:24:19.032 INFO:teuthology.orchestra.run.smithi133.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 218 kB/s | 16 kB 00:00 2024-04-02T11:24:19.074 INFO:teuthology.orchestra.run.smithi133.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-02T11:24:19.099 INFO:teuthology.orchestra.run.smithi133.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 976 kB/s | 90 kB 00:00 2024-04-02T11:24:19.125 INFO:teuthology.orchestra.run.smithi133.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 388 kB/s | 36 kB 00:00 2024-04-02T11:24:19.175 INFO:teuthology.orchestra.run.smithi133.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 401 kB/s | 20 kB 00:00 2024-04-02T11:24:19.202 INFO:teuthology.orchestra.run.smithi133.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-02T11:24:19.236 INFO:teuthology.orchestra.run.smithi133.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-04-02T11:24:19.239 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:19.239 INFO:teuthology.orchestra.run.smithi133.stdout:Total 15 MB/s | 84 MB 00:05 2024-04-02T11:24:19.264 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:24:19.306 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:24:19.306 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:24:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:24:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:24:21.626 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:24:21.696 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-02T11:24:21.737 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-02T11:24:21.765 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-02T11:24:21.837 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-02T11:24:21.881 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-02T11:24:21.914 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-02T11:24:22.006 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-02T11:24:22.093 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-02T11:24:22.133 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-02T11:24:22.204 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-02T11:24:22.258 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-04-02T11:24:22.297 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-04-02T11:24:22.346 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-04-02T11:24:22.400 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-04-02T11:24:22.640 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-04-02T11:24:22.684 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-04-02T11:24:22.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-04-02T11:24:22.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-04-02T11:24:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-04-02T11:24:22.879 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-04-02T11:24:22.914 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-02T11:24:22.947 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-04-02T11:24:22.988 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-04-02T11:24:23.023 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-04-02T11:24:23.066 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-04-02T11:24:23.156 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-04-02T11:24:23.197 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-04-02T11:24:23.269 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-04-02T11:24:23.406 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-04-02T11:24:23.532 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-02T11:24:23.730 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9. 31/36 2024-04-02T11:24:23.760 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 32/36 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 32/36 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T11:24:23.785 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:26.876 INFO:teuthology.orchestra.run.smithi138.stdout:(11/11): ceph-test-19.0.0-2589.g38caedd8.el9.x8 2.9 MB/s | 87 MB 00:29 2024-04-02T11:24:26.879 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:26.879 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.9 MB/s | 90 MB 00:30 2024-04-02T11:24:26.919 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:24:26.944 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:24:26.944 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:24:27.585 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:24:27.585 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:24:28.585 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:24:28.678 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/11 2024-04-02T11:24:28.716 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/11 2024-04-02T11:24:28.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-02T11:24:28.797 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libasan8-13.2.1-6.3.el9.x86_64 3/11 2024-04-02T11:24:28.828 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 4/11 2024-04-02T11:24:28.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/11 2024-04-02T11:24:28.918 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 6/11 2024-04-02T11:24:28.954 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 7/11 2024-04-02T11:24:28.996 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 8/11 2024-04-02T11:24:29.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.6-16.el9.x86_64 9/11 2024-04-02T11:24:29.068 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 10/11 2024-04-02T11:24:34.535 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 33/36 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 33/36 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T11:24:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:35.102 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 34/36 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 34/36 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T11:24:35.126 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:35.369 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 35/36 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 35/36 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T11:24:35.393 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:35.458 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 36/36 2024-04-02T11:24:40.671 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 10/11 2024-04-02T11:24:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 11/11 2024-04-02T11:24:41.984 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 36/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 1/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 2/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 3/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 4/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 5/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9. 6/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-04-02T11:24:41.985 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-02T11:24:41.986 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-02T11:24:41.987 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-02T11:24:41.987 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-02T11:24:41.988 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:42.738 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T11:24:42.739 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T11:24:42.740 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T11:24:42.741 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T11:24:42.741 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:42.741 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:42.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-base 2024-04-02T11:24:43.597 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:27 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:43.701 INFO:teuthology.orchestra.run.smithi133.stdout:Package ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:24:43.755 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:43.757 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:24:43.757 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:43.821 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install cephadm 2024-04-02T11:24:44.398 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:44.548 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:44.549 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:44.549 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:24:44.549 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:44.549 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:24:44.549 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 762 k 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 762 k 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 768 k 2024-04-02T11:24:44.550 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:24:44.842 INFO:teuthology.orchestra.run.smithi133.stdout:cephadm-19.0.0-2589.g38caedd8.el9.noarch.rpm 2.6 MB/s | 762 kB 00:00 2024-04-02T11:24:44.842 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:44.842 INFO:teuthology.orchestra.run.smithi133.stdout:Total 2.5 MB/s | 762 kB 00:00 2024-04-02T11:24:44.843 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:24:44.848 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:24:44.848 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:24:44.853 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:24:44.854 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:24:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:24:45.309 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:24:45.397 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:24:45.805 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:24:46.101 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:24:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:46.102 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:24:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:24:46.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:46.102 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:46.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T11:24:46.862 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:30 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:47.015 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:47.016 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:47.016 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repo Size 2024-04-02T11:24:47.016 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 157 k 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 157 k 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 640 k 2024-04-02T11:24:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 11/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 1/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 2/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 7/11 2024-04-02T11:24:47.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.6-16.el9.x86_64 8/11 2024-04-02T11:24:47.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libasan8-13.2.1-6.3.el9.x86_64 9/11 2024-04-02T11:24:47.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 10/11 2024-04-02T11:24:47.547 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-immutable-object-cache-19.0.0-2589.g38caed 297 kB/s | 157 kB 00:00 2024-04-02T11:24:47.547 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:47.548 INFO:teuthology.orchestra.run.smithi133.stdout:Total 296 kB/s | 157 kB 00:00 2024-04-02T11:24:47.548 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:24:47.557 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:24:47.557 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:24:47.609 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:24:47.609 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:24:47.753 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:24:47.815 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 11/11 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.6-16.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: libasan8-13.2.1-6.3.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: libubsan-11.4.1-3.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T11:24:47.834 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T11:24:47.835 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T11:24:47.835 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T11:24:47.835 INFO:teuthology.orchestra.run.smithi138.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-04-02T11:24:47.835 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:24:47.835 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:24:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:24:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T11:24:47.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:48.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-04-02T11:24:48.698 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:22 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:24:48.853 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout:========================================================================================= 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout:========================================================================================= 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 6.5 k 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 2.2 M 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 1.6 M 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 249 k 2024-04-02T11:24:48.856 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 5.0 M 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 71 M 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T11:24:48.857 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T11:24:48.858 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout:========================================================================================= 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout:Install 36 Packages 2024-04-02T11:24:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:24:48.860 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 84 M 2024-04-02T11:24:48.860 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 922 M 2024-04-02T11:24:48.860 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:24:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:24:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:48.875 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:24:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:24:48.876 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:48.876 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:49.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr 2024-04-02T11:24:49.406 INFO:teuthology.orchestra.run.smithi138.stdout:(1/36): ceph-19.0.0-2589.g38caedd8.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-04-02T11:24:49.689 INFO:teuthology.orchestra.run.smithi138.stdout:(2/36): ceph-mds-19.0.0-2589.g38caedd8.el9.x86_ 3.8 MB/s | 2.2 MB 00:00 2024-04-02T11:24:49.699 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:33 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:49.798 INFO:teuthology.orchestra.run.smithi133.stdout:Package ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:24:49.850 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:49.851 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:24:49.851 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:49.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T11:24:50.157 INFO:teuthology.orchestra.run.smithi138.stdout:(3/36): ceph-mgr-19.0.0-2589.g38caedd8.el9.x86_ 1.5 MB/s | 1.6 MB 00:01 2024-04-02T11:24:50.348 INFO:teuthology.orchestra.run.smithi138.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2589.g38ca 1.3 MB/s | 249 kB 00:00 2024-04-02T11:24:50.472 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:34 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:50.623 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:50.624 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-04-02T11:24:50.624 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:24:50.624 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 3.5 M 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 34 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 16 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T11:24:50.625 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:Install 11 Packages 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 7.6 M 2024-04-02T11:24:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 100 M 2024-04-02T11:24:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:24:50.632 INFO:teuthology.orchestra.run.smithi138.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 659 kB/s | 187 kB 00:00 2024-04-02T11:24:50.716 INFO:teuthology.orchestra.run.smithi138.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-04-02T11:24:50.782 INFO:teuthology.orchestra.run.smithi138.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 474 kB/s | 32 kB 00:00 2024-04-02T11:24:50.850 INFO:teuthology.orchestra.run.smithi138.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 708 kB/s | 47 kB 00:00 2024-04-02T11:24:50.925 INFO:teuthology.orchestra.run.smithi138.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 516 kB/s | 39 kB 00:00 2024-04-02T11:24:51.025 INFO:teuthology.orchestra.run.smithi138.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2024-04-02T11:24:51.242 INFO:teuthology.orchestra.run.smithi138.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 5.3 MB/s | 1.1 MB 00:00 2024-04-02T11:24:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-04-02T11:24:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 1.8 MB/s | 124 kB 00:00 2024-04-02T11:24:51.442 INFO:teuthology.orchestra.run.smithi133.stdout:(1/11): ceph-grafana-dashboards-19.0.0-2589.g38 86 kB/s | 34 kB 00:00 2024-04-02T11:24:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:(14/36): ceph-mon-19.0.0-2589.g38caedd8.el9.x86 2.5 MB/s | 5.0 MB 00:02 2024-04-02T11:24:51.459 INFO:teuthology.orchestra.run.smithi133.stdout:(2/11): ceph-prometheus-alerts-19.0.0-2589.g38c 40 kB/s | 16 kB 00:00 2024-04-02T11:24:51.468 INFO:teuthology.orchestra.run.smithi138.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.8 MB/s | 187 kB 00:00 2024-04-02T11:24:51.594 INFO:teuthology.orchestra.run.smithi138.stdout:(16/36): python3-requests-2.25.1-8.el9.noarch.r 833 kB/s | 125 kB 00:00 2024-04-02T11:24:51.644 INFO:teuthology.orchestra.run.smithi138.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-04-02T11:24:51.718 INFO:teuthology.orchestra.run.smithi133.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 863 kB/s | 238 kB 00:00 2024-04-02T11:24:51.784 INFO:teuthology.orchestra.run.smithi133.stdout:(4/11): ceph-mgr-dashboard-19.0.0-2589.g38caedd 4.8 MB/s | 3.5 MB 00:00 2024-04-02T11:24:51.859 INFO:teuthology.orchestra.run.smithi133.stdout:(5/11): protobuf-compiler-3.14.0-13.el9.x86_64. 2.1 MB/s | 863 kB 00:00 2024-04-02T11:24:51.961 INFO:teuthology.orchestra.run.smithi138.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-04-02T11:24:51.985 INFO:teuthology.orchestra.run.smithi133.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-04-02T11:24:52.012 INFO:teuthology.orchestra.run.smithi138.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 362 kB/s | 151 kB 00:00 2024-04-02T11:24:52.218 INFO:teuthology.orchestra.run.smithi133.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 620 kB/s | 144 kB 00:00 2024-04-02T11:24:52.252 INFO:teuthology.orchestra.run.smithi133.stdout:(8/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.0 MB/s | 548 kB 00:00 2024-04-02T11:24:52.287 INFO:teuthology.orchestra.run.smithi138.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-04-02T11:24:52.302 INFO:teuthology.orchestra.run.smithi133.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 367 kB/s | 31 kB 00:00 2024-04-02T11:24:52.344 INFO:teuthology.orchestra.run.smithi133.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-04-02T11:24:52.345 INFO:teuthology.orchestra.run.smithi138.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 448 kB/s | 172 kB 00:00 2024-04-02T11:24:52.372 INFO:teuthology.orchestra.run.smithi138.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-04-02T11:24:52.447 INFO:teuthology.orchestra.run.smithi138.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 174 kB/s | 18 kB 00:00 2024-04-02T11:24:52.472 INFO:teuthology.orchestra.run.smithi138.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-04-02T11:24:52.531 INFO:teuthology.orchestra.run.smithi138.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-04-02T11:24:52.556 INFO:teuthology.orchestra.run.smithi138.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 234 kB/s | 20 kB 00:00 2024-04-02T11:24:52.569 INFO:teuthology.orchestra.run.smithi133.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.9 MB/s | 2.0 MB 00:00 2024-04-02T11:24:52.572 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:52.572 INFO:teuthology.orchestra.run.smithi133.stdout:Total 3.9 MB/s | 7.6 MB 00:01 2024-04-02T11:24:52.591 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:24:52.607 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:24:52.608 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:24:52.657 INFO:teuthology.orchestra.run.smithi138.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 369 kB/s | 46 kB 00:00 2024-04-02T11:24:52.682 INFO:teuthology.orchestra.run.smithi138.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 628 kB/s | 79 kB 00:00 2024-04-02T11:24:52.795 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:24:52.796 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:24:52.807 INFO:teuthology.orchestra.run.smithi138.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2024-04-02T11:24:52.833 INFO:teuthology.orchestra.run.smithi138.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-02T11:24:52.891 INFO:teuthology.orchestra.run.smithi138.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-04-02T11:24:53.159 INFO:teuthology.orchestra.run.smithi138.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 110 kB/s | 36 kB 00:00 2024-04-02T11:24:53.184 INFO:teuthology.orchestra.run.smithi138.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 786 kB/s | 230 kB 00:00 2024-04-02T11:24:53.280 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:24:53.284 INFO:teuthology.orchestra.run.smithi138.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-04-02T11:24:53.358 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-02T11:24:53.398 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-02T11:24:53.435 INFO:teuthology.orchestra.run.smithi138.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 80 kB/s | 20 kB 00:00 2024-04-02T11:24:53.506 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-02T11:24:53.630 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-02T11:24:53.676 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-02T11:24:53.759 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-02T11:24:53.810 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-02T11:24:53.839 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-02T11:24:53.871 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9 9/11 2024-04-02T11:24:54.552 INFO:teuthology.orchestra.run.smithi138.stdout:(36/36): ceph-osd-19.0.0-2589.g38caedd8.el9.x86 15 MB/s | 71 MB 00:04 2024-04-02T11:24:54.556 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:54.556 INFO:teuthology.orchestra.run.smithi138.stdout:Total 15 MB/s | 84 MB 00:05 2024-04-02T11:24:54.581 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:24:54.624 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:24:54.624 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:24:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el 10/11 2024-04-02T11:24:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 11/11 2024-04-02T11:24:55.347 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:24:55.347 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:24:56.252 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 11/11 2024-04-02T11:24:56.252 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el 1/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 2/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9 3/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-02T11:24:56.253 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T11:24:56.581 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:56.582 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:24:56.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T11:24:56.963 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:24:57.021 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-02T11:24:57.061 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-02T11:24:57.098 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-02T11:24:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-02T11:24:57.214 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-02T11:24:57.247 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-02T11:24:57.339 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-02T11:24:57.376 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:41 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:24:57.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-02T11:24:57.467 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-02T11:24:57.531 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:24:57.532 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================ 2024-04-02T11:24:57.532 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:24:57.532 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================ 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 7.4 M 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T11:24:57.533 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================ 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout:Install 12 Packages 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:24:57.534 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 37 M 2024-04-02T11:24:57.535 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 209 M 2024-04-02T11:24:57.535 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:24:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-02T11:24:57.593 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-04-02T11:24:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-04-02T11:24:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-04-02T11:24:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-04-02T11:24:57.934 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-04-02T11:24:57.978 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-04-02T11:24:58.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-04-02T11:24:58.044 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-04-02T11:24:58.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-04-02T11:24:58.150 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-04-02T11:24:58.185 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-02T11:24:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-04-02T11:24:58.226 INFO:teuthology.orchestra.run.smithi133.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 143 kB/s | 32 kB 00:00 2024-04-02T11:24:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-04-02T11:24:58.276 INFO:teuthology.orchestra.run.smithi133.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 342 kB/s | 17 kB 00:00 2024-04-02T11:24:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-04-02T11:24:58.329 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-04-02T11:24:58.376 INFO:teuthology.orchestra.run.smithi133.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 343 kB/s | 34 kB 00:00 2024-04-02T11:24:58.417 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-04-02T11:24:58.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-04-02T11:24:58.532 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-04-02T11:24:58.594 INFO:teuthology.orchestra.run.smithi133.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.1 MB/s | 3.0 MB 00:00 2024-04-02T11:24:58.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-04-02T11:24:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-02T11:24:58.794 INFO:teuthology.orchestra.run.smithi133.stdout:(5/12): ceph-mgr-diskprediction-local-19.0.0-25 9.4 MB/s | 7.4 MB 00:00 2024-04-02T11:24:58.978 INFO:teuthology.orchestra.run.smithi133.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 7.9 MB/s | 4.7 MB 00:00 2024-04-02T11:24:58.983 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9. 31/36 2024-04-02T11:24:59.008 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 32/36 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 32/36 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T11:24:59.034 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:24:59.086 INFO:teuthology.orchestra.run.smithi133.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 748 kB/s | 217 kB 00:00 2024-04-02T11:24:59.203 INFO:teuthology.orchestra.run.smithi133.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 4.8 MB 00:00 2024-04-02T11:24:59.245 INFO:teuthology.orchestra.run.smithi133.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-04-02T11:24:59.304 INFO:teuthology.orchestra.run.smithi133.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 195 kB 00:00 2024-04-02T11:24:59.462 INFO:teuthology.orchestra.run.smithi133.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-04-02T11:24:59.829 INFO:teuthology.orchestra.run.smithi133.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-02T11:24:59.831 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:24:59.832 INFO:teuthology.orchestra.run.smithi133.stdout:Total 16 MB/s | 37 MB 00:02 2024-04-02T11:24:59.872 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:24:59.895 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:24:59.895 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:00.254 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:00.254 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:00.961 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:01.044 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T11:25:01.074 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T11:25:01.168 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T11:25:01.211 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T11:25:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T11:25:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T11:25:01.496 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T11:25:02.163 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T11:25:02.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T11:25:03.507 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T11:25:04.530 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T11:25:04.559 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 12/12 2024-04-02T11:25:07.070 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 12/12 2024-04-02T11:25:07.070 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 1/12 2024-04-02T11:25:07.070 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T11:25:07.071 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T11:25:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:07.510 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:07.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr-rook 2024-04-02T11:25:08.480 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:52 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:08.631 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 49 k 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T11:25:08.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout:Install 13 Packages 2024-04-02T11:25:08.633 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:08.634 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 2.0 M 2024-04-02T11:25:08.634 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 26 M 2024-04-02T11:25:08.634 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:08.800 INFO:teuthology.orchestra.run.smithi133.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 208 kB/s | 24 kB 00:00 2024-04-02T11:25:08.854 INFO:teuthology.orchestra.run.smithi133.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 105 kB/s | 18 kB 00:00 2024-04-02T11:25:08.901 INFO:teuthology.orchestra.run.smithi133.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 174 kB 00:00 2024-04-02T11:25:08.934 INFO:teuthology.orchestra.run.smithi133.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-04-02T11:25:08.960 INFO:teuthology.orchestra.run.smithi133.stdout:(5/13): ceph-mgr-rook-19.0.0-2589.g38caedd8.el9 178 kB/s | 49 kB 00:00 2024-04-02T11:25:08.993 INFO:teuthology.orchestra.run.smithi133.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.2 MB/s | 211 kB 00:00 2024-04-02T11:25:09.018 INFO:teuthology.orchestra.run.smithi133.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 511 kB/s | 43 kB 00:00 2024-04-02T11:25:09.186 INFO:teuthology.orchestra.run.smithi133.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 90 kB/s | 15 kB 00:00 2024-04-02T11:25:09.211 INFO:teuthology.orchestra.run.smithi133.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 129 kB/s | 32 kB 00:00 2024-04-02T11:25:09.286 INFO:teuthology.orchestra.run.smithi133.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 759 kB/s | 203 kB 00:00 2024-04-02T11:25:09.386 INFO:teuthology.orchestra.run.smithi133.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.1 MB/s | 1.0 MB 00:00 2024-04-02T11:25:09.411 INFO:teuthology.orchestra.run.smithi133.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-04-02T11:25:09.436 INFO:teuthology.orchestra.run.smithi133.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 598 kB/s | 90 kB 00:00 2024-04-02T11:25:09.441 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:09.441 INFO:teuthology.orchestra.run.smithi133.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2024-04-02T11:25:09.441 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:09.454 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:09.454 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:09.544 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:09.544 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:09.850 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 33/36 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 33/36 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T11:25:09.877 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:10.235 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T11:25:10.296 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T11:25:10.360 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T11:25:10.397 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T11:25:10.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 34/36 2024-04-02T11:25:10.432 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 34/36 2024-04-02T11:25:10.433 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:10.433 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T11:25:10.433 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T11:25:10.433 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T11:25:10.433 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:10.446 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T11:25:10.549 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T11:25:10.650 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T11:25:10.677 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 35/36 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 35/36 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T11:25:10.700 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:10.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T11:25:10.767 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 36/36 2024-04-02T11:25:11.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T11:25:11.487 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T11:25:11.520 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T11:25:11.568 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T11:25:11.590 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 13/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 13/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 1/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T11:25:12.380 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T11:25:12.381 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T11:25:12.748 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T11:25:12.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T11:25:12.750 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T11:25:12.750 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T11:25:12.750 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:12.750 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:12.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T11:25:13.563 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:57 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:13.725 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:13.725 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:13.725 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:25:13.725 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 141 k 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================== 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout:Install 6 Packages 2024-04-02T11:25:13.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:13.727 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 6.8 M 2024-04-02T11:25:13.727 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 33 M 2024-04-02T11:25:13.727 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:14.013 INFO:teuthology.orchestra.run.smithi133.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-02T11:25:14.114 INFO:teuthology.orchestra.run.smithi133.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-04-02T11:25:14.239 INFO:teuthology.orchestra.run.smithi133.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2589.g38caedd8.e 338 kB/s | 141 kB 00:00 2024-04-02T11:25:14.631 INFO:teuthology.orchestra.run.smithi133.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 117 kB/s | 58 kB 00:00 2024-04-02T11:25:14.681 INFO:teuthology.orchestra.run.smithi133.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 122 kB/s | 51 kB 00:00 2024-04-02T11:25:14.748 INFO:teuthology.orchestra.run.smithi133.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 749 kB/s | 548 kB 00:00 2024-04-02T11:25:14.752 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:14.752 INFO:teuthology.orchestra.run.smithi133.stdout:Total 6.6 MB/s | 6.8 MB 00:01 2024-04-02T11:25:14.764 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:14.774 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:14.774 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:14.862 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:14.879 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:15.180 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:15.345 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T11:25:15.390 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T11:25:15.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T11:25:15.984 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T11:25:16.047 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T11:25:16.070 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 6/6 2024-04-02T11:25:16.890 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 6/6 2024-04-02T11:25:16.891 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/6 2024-04-02T11:25:16.891 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T11:25:16.891 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T11:25:16.891 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T11:25:16.891 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T11:25:17.196 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T11:25:17.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T11:25:17.197 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T11:25:17.197 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:17.197 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:17.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-fuse 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 36/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 1/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 2/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 3/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 4/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 5/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9. 6/36 2024-04-02T11:25:17.985 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-04-02T11:25:17.986 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-02T11:25:17.987 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-02T11:25:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-02T11:25:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-02T11:25:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-02T11:25:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-02T11:25:17.988 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-02T11:25:18.037 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:02 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:18.190 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 871 k 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout:Install 2 Packages 2024-04-02T11:25:18.191 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:18.192 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 950 k 2024-04-02T11:25:18.192 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 4.0 M 2024-04-02T11:25:18.192 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:18.325 INFO:teuthology.orchestra.run.smithi133.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 595 kB/s | 79 kB 00:00 2024-04-02T11:25:18.775 INFO:teuthology.orchestra.run.smithi133.stdout:(2/2): ceph-fuse-19.0.0-2589.g38caedd8.el9.x86_ 1.5 MB/s | 871 kB 00:00 2024-04-02T11:25:18.776 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:18.776 INFO:teuthology.orchestra.run.smithi133.stdout:Total 1.6 MB/s | 950 kB 00:00 2024-04-02T11:25:18.777 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:18.798 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:18.798 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:18.871 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:18.871 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T11:25:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T11:25:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T11:25:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T11:25:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:18.987 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:19.054 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:19.201 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T11:25:19.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-04-02T11:25:19.267 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2/2 2024-04-02T11:25:19.804 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:53 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:19.903 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:19.954 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:19.955 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:25:19.955 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:20.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-04-02T11:25:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2/2 2024-04-02T11:25:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/2 2024-04-02T11:25:20.404 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T11:25:20.404 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:20.404 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:20.404 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T11:25:20.405 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:20.405 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:20.583 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:54 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:20.698 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-volume 2024-04-02T11:25:20.734 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:20.734 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 762 k 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:25:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:20.736 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 762 k 2024-04-02T11:25:20.736 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 768 k 2024-04-02T11:25:20.736 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:20.990 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-19.0.0-2589.g38caedd8.el9.noarch.rpm 2.9 MB/s | 762 kB 00:00 2024-04-02T11:25:20.991 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:20.991 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.9 MB/s | 762 kB 00:00 2024-04-02T11:25:20.991 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:20.996 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:20.996 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:21.001 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:21.002 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:21.078 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:21.282 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:05 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:21.435 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 264 k 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:21.436 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:21.437 INFO:teuthology.orchestra.run.smithi133.stdout:Install 2 Packages 2024-04-02T11:25:21.437 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:21.437 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 332 k 2024-04-02T11:25:21.437 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 1.5 M 2024-04-02T11:25:21.437 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:21.507 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:25:21.562 INFO:teuthology.orchestra.run.smithi133.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 551 kB/s | 69 kB 00:00 2024-04-02T11:25:21.595 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:25:21.813 INFO:teuthology.orchestra.run.smithi133.stdout:(2/2): ceph-volume-19.0.0-2589.g38caedd8.el9.no 704 kB/s | 264 kB 00:00 2024-04-02T11:25:21.814 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:21.814 INFO:teuthology.orchestra.run.smithi133.stdout:Total 881 kB/s | 332 kB 00:00 2024-04-02T11:25:21.814 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:21.822 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:21.822 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:21.859 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:21.859 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:25:22.083 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:22.252 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/1 2024-04-02T11:25:22.252 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:22.252 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:22.252 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:22.252 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:22.253 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:22.270 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-02T11:25:22.307 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2/2 2024-04-02T11:25:22.323 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2/2 2024-04-02T11:25:22.323 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:22.324 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T11:25:22.324 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:22.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T11:25:23.055 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:57 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:23.120 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 1/2 2024-04-02T11:25:23.205 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:23.205 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 157 k 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 157 k 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 640 k 2024-04-02T11:25:23.206 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:23.433 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-19.0.0-2589.g38caed 695 kB/s | 157 kB 00:00 2024-04-02T11:25:23.434 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:23.434 INFO:teuthology.orchestra.run.smithi138.stdout:Total 690 kB/s | 157 kB 00:00 2024-04-02T11:25:23.435 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:23.443 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:23.443 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:23.455 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:25:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:25:23.731 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:23.731 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T11:25:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:23.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librados-devel 2024-04-02T11:25:24.331 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:08 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:24.481 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:24.481 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repo Size 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 125 k 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 125 k 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 445 k 2024-04-02T11:25:24.482 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8. 1/1 2024-04-02T11:25:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:24.711 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:24.711 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:24.802 INFO:teuthology.orchestra.run.smithi133.stdout:librados-devel-19.0.0-2589.g38caedd8.el9.x86_64 391 kB/s | 125 kB 00:00 2024-04-02T11:25:24.802 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:24.803 INFO:teuthology.orchestra.run.smithi133.stdout:Total 389 kB/s | 125 kB 00:00 2024-04-02T11:25:24.803 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:24.809 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:24.809 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:24.865 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:24.865 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:24.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-04-02T11:25:25.015 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:25.108 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:25.513 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:59 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:25.584 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:25.610 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:25.662 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:25.663 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:25:25.663 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:25.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T11:25:25.877 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:25.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:25.878 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:25.878 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:25.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:25.878 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:26.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install libcephfs2 2024-04-02T11:25:26.285 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:00 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:26.435 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:26.436 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-02T11:25:26.436 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:26.436 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-02T11:25:26.436 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:26.436 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 3.5 M 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 34 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 16 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout:Install 11 Packages 2024-04-02T11:25:26.437 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:26.438 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.6 M 2024-04-02T11:25:26.438 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 100 M 2024-04-02T11:25:26.438 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:26.699 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:10 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:26.801 INFO:teuthology.orchestra.run.smithi133.stdout:Package libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:26.840 INFO:teuthology.orchestra.run.smithi138.stdout:(1/11): ceph-grafana-dashboards-19.0.0-2589.g38 156 kB/s | 34 kB 00:00 2024-04-02T11:25:26.856 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:(2/11): ceph-prometheus-alerts-19.0.0-2589.g38c 70 kB/s | 16 kB 00:00 2024-04-02T11:25:26.858 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:26.858 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:26.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install libcephfs-devel 2024-04-02T11:25:26.974 INFO:teuthology.orchestra.run.smithi138.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.7 MB/s | 238 kB 00:00 2024-04-02T11:25:27.091 INFO:teuthology.orchestra.run.smithi138.stdout:(4/11): ceph-mgr-dashboard-19.0.0-2589.g38caedd 7.6 MB/s | 3.5 MB 00:00 2024-04-02T11:25:27.167 INFO:teuthology.orchestra.run.smithi138.stdout:(5/11): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-04-02T11:25:27.275 INFO:teuthology.orchestra.run.smithi138.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.8 MB/s | 548 kB 00:00 2024-04-02T11:25:27.292 INFO:teuthology.orchestra.run.smithi138.stdout:(7/11): grpc-data-1.46.7-10.el9.noarch.rpm 97 kB/s | 19 kB 00:00 2024-04-02T11:25:27.376 INFO:teuthology.orchestra.run.smithi138.stdout:(8/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.8 MB/s | 2.0 MB 00:00 2024-04-02T11:25:27.401 INFO:teuthology.orchestra.run.smithi138.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 285 kB/s | 31 kB 00:00 2024-04-02T11:25:27.426 INFO:teuthology.orchestra.run.smithi138.stdout:(10/11): python3-grpcio-tools-1.46.7-10.el9.x86 1.1 MB/s | 144 kB 00:00 2024-04-02T11:25:27.452 INFO:teuthology.orchestra.run.smithi138.stdout:(11/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-04-02T11:25:27.456 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:27.456 INFO:teuthology.orchestra.run.smithi138.stdout:Total 7.5 MB/s | 7.6 MB 00:01 2024-04-02T11:25:27.473 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:27.488 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:11 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:27.489 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:27.490 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:27.639 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:27.639 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repo Size 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 32 k 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:25:27.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:27.641 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 32 k 2024-04-02T11:25:27.641 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 142 k 2024-04-02T11:25:27.641 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:27.673 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:27.673 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:27.888 INFO:teuthology.orchestra.run.smithi133.stdout:libcephfs-devel-19.0.0-2589.g38caedd8.el9.x86_6 128 kB/s | 32 kB 00:00 2024-04-02T11:25:27.889 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:27.889 INFO:teuthology.orchestra.run.smithi133.stdout:Total 127 kB/s | 32 kB 00:00 2024-04-02T11:25:27.889 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:27.892 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:27.893 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:27.926 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:27.926 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:28.013 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:28.120 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:28.163 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:28.233 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-04-02T11:25:28.263 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-04-02T11:25:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-04-02T11:25:28.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-04-02T11:25:28.527 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:28.584 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-04-02T11:25:28.667 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-04-02T11:25:28.711 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-04-02T11:25:28.741 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-04-02T11:25:28.773 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9 9/11 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:28.801 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:29.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librados2 2024-04-02T11:25:29.567 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:13 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:29.667 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:29.722 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:29.723 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:29.723 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:29.796 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librbd1 2024-04-02T11:25:29.816 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el 10/11 2024-04-02T11:25:29.866 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 11/11 2024-04-02T11:25:30.364 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:14 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:30.463 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:30.516 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:30.517 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:30.517 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:30.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rados 2024-04-02T11:25:31.153 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:15 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:31.225 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 11/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el 1/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noa 2/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9 3/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-04-02T11:25:31.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-04-02T11:25:31.251 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:31.304 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:31.305 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:31.305 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:31.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rgw 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T11:25:31.551 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:31.552 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:31.801 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T11:25:31.939 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:15 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:32.039 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:32.091 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:32.092 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:32.093 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:32.168 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-cephfs 2024-04-02T11:25:32.367 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:06 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:32.519 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 7.4 M 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T11:25:32.521 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================================ 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-04-02T11:25:32.522 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:32.523 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 37 M 2024-04-02T11:25:32.523 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 209 M 2024-04-02T11:25:32.523 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:32.725 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:16 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:32.825 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:32.838 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 242 kB/s | 32 kB 00:00 2024-04-02T11:25:32.872 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 530 kB/s | 17 kB 00:00 2024-04-02T11:25:32.876 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:32.877 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:32.877 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:32.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rbd 2024-04-02T11:25:32.955 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-02T11:25:32.981 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 319 kB/s | 34 kB 00:00 2024-04-02T11:25:33.173 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2024-04-02T11:25:33.248 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-04-02T11:25:33.273 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-04-02T11:25:33.457 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-02T11:25:33.513 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:17 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:33.613 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:25:33.676 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-02T11:25:33.677 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-02T11:25:33.677 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:33.677 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-04-02T11:25:33.677 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:33.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-fuse 2024-04-02T11:25:33.800 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-04-02T11:25:34.308 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:18 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:34.459 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-04-02T11:25:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 90 k 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 90 k 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 313 k 2024-04-02T11:25:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:34.525 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 4.1 MB/s | 7.4 MB 00:01 2024-04-02T11:25:34.527 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:34.528 INFO:teuthology.orchestra.run.smithi138.stdout:Total 18 MB/s | 37 MB 00:02 2024-04-02T11:25:34.570 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:34.593 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:34.594 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:34.793 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-fuse-19.0.0-2589.g38caedd8.el9.x86_64.rpm 271 kB/s | 90 kB 00:00 2024-04-02T11:25:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:Total 270 kB/s | 90 kB 00:00 2024-04-02T11:25:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:34.802 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:34.802 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:34.861 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:34.861 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:34.948 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:34.948 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:35.000 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:35.100 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:35.516 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:35.646 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:35.731 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T11:25:35.771 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T11:25:35.797 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:35.798 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:35.798 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:35.798 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:35.798 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:35.798 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T11:25:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T11:25:36.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-mirror 2024-04-02T11:25:36.086 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T11:25:36.116 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T11:25:36.198 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T11:25:36.624 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:20 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:36.780 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:36.780 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repo Size 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.4 M 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 3.4 M 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 29 M 2024-04-02T11:25:36.781 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:36.833 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T11:25:36.945 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T11:25:37.179 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-mirror-19.0.0-2589.g38caedd8.el9.x86_64.rpm 8.6 MB/s | 3.4 MB 00:00 2024-04-02T11:25:37.180 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:37.180 INFO:teuthology.orchestra.run.smithi133.stdout:Total 8.5 MB/s | 3.4 MB 00:00 2024-04-02T11:25:37.180 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:37.190 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:37.191 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:37.266 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:37.266 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:37.836 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:37.897 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:37.921 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:37.921 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:37.921 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T11:25:37.922 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T11:25:37.922 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T11:25:37.922 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:38.184 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T11:25:39.225 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:39.250 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 12/12 2024-04-02T11:25:39.499 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-nbd 2024-04-02T11:25:40.084 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:24 ago on Tue 02 Apr 2024 11:23:16 AM UTC. 2024-04-02T11:25:40.238 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 176 k 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-04-02T11:25:40.239 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2024-04-02T11:25:40.240 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:40.240 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 176 k 2024-04-02T11:25:40.240 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 673 k 2024-04-02T11:25:40.240 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-02T11:25:40.465 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-nbd-19.0.0-2589.g38caedd8.el9.x86_64.rpm 779 kB/s | 176 kB 00:00 2024-04-02T11:25:40.466 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:40.466 INFO:teuthology.orchestra.run.smithi133.stdout:Total 774 kB/s | 176 kB 00:00 2024-04-02T11:25:40.466 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-02T11:25:40.476 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-02T11:25:40.476 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-02T11:25:40.533 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-02T11:25:40.533 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-02T11:25:40.684 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-02T11:25:40.789 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:41.267 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:41.539 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:25:41.539 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:41.540 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-02T11:25:41.540 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:25:41.540 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:25:41.540 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-02T11:25:41.803 DEBUG:teuthology.parallel:result is None 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 12/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2589.g38cae 1/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T11:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T11:25:41.901 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T11:25:41.901 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T11:25:41.901 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T11:25:41.901 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T11:25:42.310 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T11:25:42.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T11:25:42.311 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:42.312 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:42.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-04-02T11:25:43.313 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:17 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:43.474 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:43.475 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 49 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T11:25:43.476 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-04-02T11:25:43.477 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:43.478 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.0 M 2024-04-02T11:25:43.478 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-04-02T11:25:43.478 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:43.843 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 199 kB/s | 24 kB 00:00 2024-04-02T11:25:43.860 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 130 kB/s | 18 kB 00:00 2024-04-02T11:25:43.918 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-04-02T11:25:43.944 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-04-02T11:25:43.969 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-04-02T11:25:43.986 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2024-04-02T11:25:44.003 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): ceph-mgr-rook-19.0.0-2589.g38caedd8.el9 174 kB/s | 49 kB 00:00 2024-04-02T11:25:44.245 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 117 kB/s | 32 kB 00:00 2024-04-02T11:25:44.271 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 53 kB/s | 15 kB 00:00 2024-04-02T11:25:44.438 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 354 kB/s | 59 kB 00:00 2024-04-02T11:25:44.488 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 418 kB/s | 203 kB 00:00 2024-04-02T11:25:44.555 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.3 MB/s | 1.0 MB 00:00 2024-04-02T11:25:44.572 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 669 kB/s | 90 kB 00:00 2024-04-02T11:25:44.576 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:44.576 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-04-02T11:25:44.577 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:44.590 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:44.590 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:44.680 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:44.680 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:45.196 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:45.367 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T11:25:45.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T11:25:45.487 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T11:25:45.525 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T11:25:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T11:25:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T11:25:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T11:25:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T11:25:46.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T11:25:46.575 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T11:25:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T11:25:46.657 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T11:25:46.679 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 13/13 2024-04-02T11:25:47.498 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 13/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 1/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T11:25:47.499 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T11:25:47.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T11:25:47.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:47.850 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:47.851 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:48.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T11:25:48.662 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:22 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:48.816 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 141 k 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:48.817 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================== 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 6.8 M 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-04-02T11:25:48.818 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:49.404 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-02T11:25:49.538 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-04-02T11:25:49.671 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2589.g38caedd8.e 316 kB/s | 141 kB 00:00 2024-04-02T11:25:49.738 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2024-04-02T11:25:49.763 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 257 kB/s | 58 kB 00:00 2024-04-02T11:25:49.809 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 371 kB/s | 51 kB 00:00 2024-04-02T11:25:49.811 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:49.812 INFO:teuthology.orchestra.run.smithi138.stdout:Total 6.8 MB/s | 6.8 MB 00:00 2024-04-02T11:25:49.825 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:49.834 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:49.834 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:49.924 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:49.924 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:50.217 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:50.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T11:25:50.458 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T11:25:50.972 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T11:25:51.060 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T11:25:51.124 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T11:25:51.154 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 6/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 6/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 1/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T11:25:52.054 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T11:25:52.349 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T11:25:52.350 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T11:25:52.350 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:52.350 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:52.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-04-02T11:25:53.195 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:27 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:53.350 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 871 k 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-02T11:25:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:53.352 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 950 k 2024-04-02T11:25:53.352 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 4.0 M 2024-04-02T11:25:53.352 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:53.477 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 632 kB/s | 79 kB 00:00 2024-04-02T11:25:53.810 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-19.0.0-2589.g38caedd8.el9.x86_ 1.9 MB/s | 871 kB 00:00 2024-04-02T11:25:53.812 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:53.812 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.0 MB/s | 950 kB 00:00 2024-04-02T11:25:53.812 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:53.835 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:53.835 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:53.908 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:53.909 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:54.248 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T11:25:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2/2 2024-04-02T11:25:54.997 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2/2 2024-04-02T11:25:54.997 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/2 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:55.310 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:55.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-volume 2024-04-02T11:25:56.183 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:30 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:56.342 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-2589.g38caedd8.el9 ceph-noarch 264 k 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:56.343 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:56.344 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-02T11:25:56.344 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:56.344 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 332 k 2024-04-02T11:25:56.345 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.5 M 2024-04-02T11:25:56.345 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:56.460 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 596 kB/s | 69 kB 00:00 2024-04-02T11:25:56.702 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-volume-19.0.0-2589.g38caedd8.el9.no 738 kB/s | 264 kB 00:00 2024-04-02T11:25:56.703 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:56.703 INFO:teuthology.orchestra.run.smithi138.stdout:Total 924 kB/s | 332 kB 00:00 2024-04-02T11:25:56.704 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:56.712 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:56.712 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:56.749 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:56.749 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:56.956 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:25:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-02T11:25:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2/2 2024-04-02T11:25:57.205 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2/2 2024-04-02T11:25:57.206 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:25:57.206 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T11:25:57.206 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 1/2 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-2589.g38caedd8.el9.noarch 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:58.273 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:25:58.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-04-02T11:25:59.201 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:33 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:25:59.351 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:25:59.352 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:59.352 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-02T11:25:59.352 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:59.352 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:25:59.352 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 125 k 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 125 k 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 445 k 2024-04-02T11:25:59.353 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:25:59.721 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-19.0.0-2589.g38caedd8.el9.x86_64 340 kB/s | 125 kB 00:00 2024-04-02T11:25:59.721 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:25:59.721 INFO:teuthology.orchestra.run.smithi138.stdout:Total 338 kB/s | 125 kB 00:00 2024-04-02T11:25:59.722 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:25:59.727 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:25:59.727 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:25:59.784 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:25:59.784 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:25:59.929 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:26:00.053 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:00.498 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:00.780 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:00.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-04-02T11:26:01.568 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:35 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:01.671 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:01.723 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:01.724 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:01.724 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:01.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-04-02T11:26:02.345 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:36 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:02.495 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:02.495 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 32 k 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 32 k 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 142 k 2024-04-02T11:26:02.496 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:26:02.661 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-19.0.0-2589.g38caedd8.el9.x86_6 193 kB/s | 32 kB 00:00 2024-04-02T11:26:02.662 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:26:02.662 INFO:teuthology.orchestra.run.smithi138.stdout:Total 191 kB/s | 32 kB 00:00 2024-04-02T11:26:02.662 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:26:02.665 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:26:02.665 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:26:02.693 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:26:02.693 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:26:02.788 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:26:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:03.297 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:03.545 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:26:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:26:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:03.546 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:03.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-04-02T11:26:04.274 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:38 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:04.374 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:04.427 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:04.428 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:04.428 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:04.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-04-02T11:26:05.051 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:39 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:05.151 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:05.204 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:05.205 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:05.205 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:05.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-04-02T11:26:05.840 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:39 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:05.939 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:05.992 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:05.993 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:05.993 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:06.058 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-04-02T11:26:06.625 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:40 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:06.725 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:06.777 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:06.778 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:06.778 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:06.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-04-02T11:26:07.397 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:41 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:07.496 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:07.550 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:07.552 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:07.552 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:07.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-04-02T11:26:08.180 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:42 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:08.279 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:19.0.0-2589.g38caedd8.el9.x86_64 is already installed. 2024-04-02T11:26:08.332 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:08.333 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-02T11:26:08.333 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:08.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-04-02T11:26:08.968 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:42 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:09.121 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:09.122 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-02T11:26:09.122 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 90 k 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 90 k 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 313 k 2024-04-02T11:26:09.123 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:26:09.332 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-19.0.0-2589.g38caedd8.el9.x86_64.rpm 431 kB/s | 90 kB 00:00 2024-04-02T11:26:09.332 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:26:09.332 INFO:teuthology.orchestra.run.smithi138.stdout:Total 428 kB/s | 90 kB 00:00 2024-04-02T11:26:09.333 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:26:09.341 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:26:09.341 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:26:09.397 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:26:09.397 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:26:09.532 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:26:09.638 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:10.009 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:10.306 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:10.306 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:10.307 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:26:10.307 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:26:10.307 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:10.307 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:10.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-04-02T11:26:11.125 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:45 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:11.279 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:11.280 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:11.280 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-02T11:26:11.280 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:11.280 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:26:11.280 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 3.4 M 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.4 M 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 29 M 2024-04-02T11:26:11.281 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:26:12.863 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-19.0.0-2589.g38caedd8.el9.x86_64.rpm 2.2 MB/s | 3.4 MB 00:01 2024-04-02T11:26:12.864 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:26:12.864 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.1 MB/s | 3.4 MB 00:01 2024-04-02T11:26:12.864 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:26:12.877 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:26:12.877 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:26:12.956 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:26:12.956 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:26:13.535 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:26:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T11:26:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:14.820 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:15.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-04-02T11:26:15.687 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:02:49 ago on Tue 02 Apr 2024 11:23:26 AM UTC. 2024-04-02T11:26:15.849 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-2589.g38caedd8.el9 ceph 176 k 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:15.850 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 176 k 2024-04-02T11:26:15.851 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 673 k 2024-04-02T11:26:15.851 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-02T11:26:16.246 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-19.0.0-2589.g38caedd8.el9.x86_64.rpm 445 kB/s | 176 kB 00:00 2024-04-02T11:26:16.246 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-02T11:26:16.247 INFO:teuthology.orchestra.run.smithi138.stdout:Total 443 kB/s | 176 kB 00:00 2024-04-02T11:26:16.247 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-02T11:26:16.257 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-02T11:26:16.257 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-02T11:26:16.315 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-02T11:26:16.316 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-02T11:26:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-02T11:26:16.578 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:17.037 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 1/1 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-2589.g38caedd8.el9.x86_64 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:17.329 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-02T11:26:17.543 DEBUG:teuthology.parallel:result is None 2024-04-02T11:26:17.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=38caedd896c6b19ed5e43655372d95dc8e78cfda 2024-04-02T11:26:17.680 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T11:26:17.705 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-2589.g38caedd8.el9 2024-04-02T11:26:17.706 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2589.g38caedd8.el9 2024-04-02T11:26:17.706 INFO:teuthology.task.install:The correct ceph version 19.0.0-2589.g38caedd8 is installed. 2024-04-02T11:26:17.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=38caedd896c6b19ed5e43655372d95dc8e78cfda 2024-04-02T11:26:17.927 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T11:26:17.948 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-2589.g38caedd8.el9 2024-04-02T11:26:17.948 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2589.g38caedd8.el9 2024-04-02T11:26:17.948 INFO:teuthology.task.install:The correct ceph version 19.0.0-2589.g38caedd8 is installed. 2024-04-02T11:26:17.950 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T11:26:17.950 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:17.950 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T11:26:17.984 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:17.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T11:26:18.016 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T11:26:18.016 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:18.016 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T11:26:18.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T11:26:18.122 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:18.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T11:26:18.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T11:26:18.216 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T11:26:18.217 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:18.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T11:26:18.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T11:26:18.312 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:18.313 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T11:26:18.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T11:26:18.406 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T11:26:18.407 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:18.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T11:26:18.434 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T11:26:18.502 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:18.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T11:26:18.529 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T11:26:18.596 INFO:teuthology.run_tasks:Running task ceph... 2024-04-02T11:26:18.678 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-02T11:26:18.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-04-02T11:26:18.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-02T11:26:18.707 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-02T11:26:18.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T11:26:18.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T11:26:18.774 INFO:tasks.ceph:Creating extra log directories... 2024-04-02T11:26:18.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T11:26:18.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T11:26:18.852 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-02T11:26:18.852 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-02T11:26:18.852 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494', 'branch': 'wip-matanb-crimson-testing-apr-2', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7635494', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-testing-apr-2'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '107e1624f2a9b72a56ee529969558944d24a2a75'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 223, 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '107e1624f2a9b72a56ee529969558944d24a2a75', 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJK+nFThnXLgUX2cmMcAQA4Ue5WcKpH70qrYfnnQQBrUA/tKnCFvCD3dWiMXtv168DqA/XIP9BUTn5gZh8XTUc=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBILXb1sCtBCsjkTFXqvIZz4kqBeJ8pjSF1jv1Hq7ql4vvYJOoCxP9hUF/7C9k5YUBVahFiqqeq05+Sz6GbqJOyU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '38caedd896c6b19ed5e43655372d95dc8e78cfda', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-04-02_11:03:04', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-02T11:26:18.853 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T11:26:18.882 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T11:26:18.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T11:26:18.937 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T11:26:18.965 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:18.965 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T11:26:19.016 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T11:26:19.017 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-04-02T11:26:19.070 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T11:26:19.070 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.070 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-02 11:25:38.189463924 +0000 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-02 11:22:33.737825098 +0000 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-02 11:22:33.737825098 +0000 2024-04-02T11:26:19.071 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-02 11:22:33.737825098 +0000 2024-04-02T11:26:19.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T11:26:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-02T11:26:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-02T11:26:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000200937 s, 2.5 MB/s 2024-04-02T11:26:19.137 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T11:26:19.192 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-02 11:25:38.189463924 +0000 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-02 11:22:34.085818452 +0000 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-02 11:22:34.085818452 +0000 2024-04-02T11:26:19.248 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-02 11:22:34.085818452 +0000 2024-04-02T11:26:19.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T11:26:19.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-02T11:26:19.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-02T11:26:19.314 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000183117 s, 2.8 MB/s 2024-04-02T11:26:19.315 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T11:26:19.370 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.426 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-02 11:25:38.190463905 +0000 2024-04-02T11:26:19.427 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-02 11:22:34.424811977 +0000 2024-04-02T11:26:19.427 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-02 11:22:34.424811977 +0000 2024-04-02T11:26:19.427 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-02 11:22:34.424811977 +0000 2024-04-02T11:26:19.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T11:26:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-02T11:26:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-02T11:26:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000221122 s, 2.3 MB/s 2024-04-02T11:26:19.493 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T11:26:19.549 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-02 11:25:38.190463905 +0000 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-02 11:22:34.765805465 +0000 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-02 11:22:34.765805465 +0000 2024-04-02T11:26:19.604 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-02 11:22:34.765805465 +0000 2024-04-02T11:26:19.604 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T11:26:19.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-02T11:26:19.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-02T11:26:19.669 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000275538 s, 1.9 MB/s 2024-04-02T11:26:19.671 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T11:26:19.726 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:19.726 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:19.726 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T11:26:19.744 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T11:26:19.744 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-02T11:26:19.800 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T11:26:19.800 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.800 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-02T11:26:19.800 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.800 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.801 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-02 11:26:13.859309518 +0000 2024-04-02T11:26:19.801 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-02 11:22:33.727768147 +0000 2024-04-02T11:26:19.801 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-02 11:22:33.727768147 +0000 2024-04-02T11:26:19.801 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-02 11:22:33.727768147 +0000 2024-04-02T11:26:19.801 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T11:26:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-02T11:26:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-02T11:26:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000492027 s, 1.0 MB/s 2024-04-02T11:26:19.869 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T11:26:19.926 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-02 11:26:13.859309518 +0000 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-02 11:22:34.056761598 +0000 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-02 11:22:34.056761598 +0000 2024-04-02T11:26:19.982 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-02 11:22:34.056761598 +0000 2024-04-02T11:26:19.983 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T11:26:20.049 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-02T11:26:20.049 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-02T11:26:20.049 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000171088 s, 3.0 MB/s 2024-04-02T11:26:20.050 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T11:26:20.106 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-02 11:26:13.860309498 +0000 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-02 11:22:34.411754530 +0000 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-02 11:22:34.411754530 +0000 2024-04-02T11:26:20.163 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-02 11:22:34.411754530 +0000 2024-04-02T11:26:20.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T11:26:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-02T11:26:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-02T11:26:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000211419 s, 2.4 MB/s 2024-04-02T11:26:20.230 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T11:26:20.287 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-02T11:26:20.343 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T11:26:20.343 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-02 11:26:13.860309498 +0000 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-02 11:22:34.755747682 +0000 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-02 11:22:34.755747682 +0000 2024-04-02T11:26:20.344 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-02 11:22:34.755747682 +0000 2024-04-02T11:26:20.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T11:26:20.408 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-02T11:26:20.409 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-02T11:26:20.409 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000190097 s, 2.7 MB/s 2024-04-02T11:26:20.410 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T11:26:20.465 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:20.465 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-02T11:26:20.465 INFO:tasks.ceph:Generating config... 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-04-02T11:26:20.467 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-04-02T11:26:20.467 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] osd class default list = * 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] osd class load list = * 2024-04-02T11:26:20.468 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-02T11:26:20.468 INFO:tasks.ceph:[client] debug ms = 1 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] ms client mode = crc 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-04-02T11:26:20.468 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-04-02T11:26:20.469 INFO:tasks.ceph:[global] ms service mode = crc 2024-04-02T11:26:20.469 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-02T11:26:20.469 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-02T11:26:20.469 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.469 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.469 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.469 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.469 INFO:tasks.ceph:Setting up mon.a... 2024-04-02T11:26:20.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-02T11:26:20.534 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-04-02T11:26:20.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-02T11:26:20.591 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T11:26:20.617 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.133')] 2024-04-02T11:26:20.618 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.133', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-04-02T11:26:20.619 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:20.619 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T11:26:20.672 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.133 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:20.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:20.761+0000 7ff14d348f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:20.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:20.761+0000 7ff14d348f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:20.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:20.761+0000 7ff14d348f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:20.762 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:20.762 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-04-02T11:26:20.762604+0000 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-04-02T11:26:20.762604+0000 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 17 (quincy) 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-04-02T11:26:20.763 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-02T11:26:20.764 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T11:26:20.779 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0fac6a5d-b779-400c-9b2f-b14c76190269... 2024-04-02T11:26:20.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T11:26:20.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T11:26:20.869 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-02T11:26:20.870 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-02T11:26:20.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-02T11:26:20.872 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = 172.21.15.133 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms client mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms cluster mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms mon client mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms mon cluster mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms mon service mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: ms service mode = crc 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-02T11:26:20.873 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.874 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug monc = 20 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: crimson alien thread cpu cores = 6-7 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: crimson osd obc lru size = 10 2024-04-02T11:26:20.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd blocked scrub grace period = 3600 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd class default list = * 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd class load list = * 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd shutdown pgref assert = True 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-02T11:26:20.876 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T11:26:20.877 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon min osdmap epochs = 50 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon osdmap full prune interval = 2 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon osdmap full prune min = 15 2024-04-02T11:26:20.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon osdmap full prune txsize = 2 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: paxos service trim min = 10 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on pool no app = False 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default crimson = True 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout: crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.879 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi133.stdout: crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi133.stdout: crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi133.stdout: crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T11:26:20.880 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.881 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-02T11:26:20.882 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = 172.21.15.133 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: ms client mode = crc 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: ms cluster mode = crc 2024-04-02T11:26:20.883 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon client mode = crc 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon cluster mode = crc 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: ms mon service mode = crc 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: ms service mode = crc 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-02T11:26:20.884 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.885 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: debug monc = 20 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: crimson alien thread cpu cores = 6-7 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: crimson osd obc lru size = 10 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: osd blocked scrub grace period = 3600 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: osd class default list = * 2024-04-02T11:26:20.886 INFO:teuthology.orchestra.run.smithi138.stdout: osd class load list = * 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: osd shutdown pgref assert = True 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-02T11:26:20.887 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T11:26:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon min osdmap epochs = 50 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune interval = 2 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune min = 15 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon osdmap full prune txsize = 2 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: paxos service trim min = 10 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on pool no app = False 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default crimson = True 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-02T11:26:20.889 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-04-02T11:26:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: crimson seastar cpu cores = 0-2 2024-04-02T11:26:20.891 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-04-02T11:26:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: crimson seastar cpu cores = 3-5 2024-04-02T11:26:20.892 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-02T11:26:20.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-02T11:26:20.957 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-02T11:26:20.957 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:20.958 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-02T11:26:20.972 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:20.972 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-02T11:26:21.026 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T11:26:21.027 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:21.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T11:26:21.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T11:26:21.106 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:21.106 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:21.161 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T11:26:21.161 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:21.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T11:26:21.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T11:26:21.201 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:21.202 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:21.256 INFO:tasks.ceph:Setting up mon nodes... 2024-04-02T11:26:21.257 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-02T11:26:21.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T11:26:21.389 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T11:26:21.392 INFO:tasks.ceph:Setting up mds nodes... 2024-04-02T11:26:21.392 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-02T11:26:21.392 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T11:26:21.443 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-02T11:26:21.460 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-02T11:26:21.511 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-02T11:26:21.526 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-02T11:26:21.527 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-04-02T11:26:21.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-02T11:26:21.553 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:21.554 INFO:tasks.ceph:role: osd.0 2024-04-02T11:26:21.554 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T11:26:21.554 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T11:26:21.628 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T11:26:21.629 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T11:26:22.121 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-02T11:26:22.122 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-02T11:26:22.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-02T11:26:22.242 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-02T11:26:22.277 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-02T11:26:22.345 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:22.345 INFO:tasks.ceph:role: osd.1 2024-04-02T11:26:22.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T11:26:22.345 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T11:26:22.419 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-02T11:26:22.420 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T11:26:22.420 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T11:26:22.420 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T11:26:22.420 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T11:26:22.881 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-02T11:26:22.883 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-02T11:26:22.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-02T11:26:22.973 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-02T11:26:23.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:23,338 seastar - Reactor backend: linux-aio 2024-04-02T11:26:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:==28945==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:23.470 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:23,470 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-02T11:26:23.470 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:23,470 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-02T11:26:23.470 INFO:teuthology.orchestra.run.smithi133.stderr:DEBUG 2024-04-02 11:26:23,471 [shard 0:main] seastar - reactor::drain 2024-04-02T11:26:24.009 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:24.009 INFO:teuthology.orchestra.run.smithi133.stderr:Suppressions used: 2024-04-02T11:26:24.009 INFO:teuthology.orchestra.run.smithi133.stderr: count bytes template 2024-04-02T11:26:24.009 INFO:teuthology.orchestra.run.smithi133.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:24.009 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:24.010 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-02T11:26:24.072 INFO:teuthology.orchestra.run.smithi133.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:24.076 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:24,077 seastar - Reactor backend: linux-aio 2024-04-02T11:26:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:==28943==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:24.150 INFO:teuthology.orchestra.run.smithi133.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9de8c9 0xb8b1390 0xb8b290c 0xb8b2b22 0xb8b2c78 0xb8b3141 0x54daf 0x34d0b40 0x34d12b7 0x5d34e75 0x5c07dbf 0x5c0907e 0x6ee1bf4 0x6ef33c7 0x6ef4864 0x6ef48e9 0x6ef49eb 0x6ef4ae3 0x6ef4cf0 0x6ef4ed6 0x6ef5225 0x6ef55c9 0x6ef57f9 0x6ef5a63 0x6ef5d30 0x6ef5f4e 0x6ef6006 0x6ef60cc 0x6ef61ad 0x5a54aa8 0x5a54bb2 0x5a54c6e 0x5a54d2a 0x5a54de6 0x5a54f33 0x5a550be 0x5a5547f 0x5a71efc 0xb8c4015 0xb8de2af 0xb97f2ad 0xb9808fb 0xb65d063 0xb65d9df 0x368b58a 0x3feaf 0x3ff5f 0x3477444 2024-04-02T11:26:24.150 INFO:teuthology.orchestra.run.smithi133.stderr:kernel callstack: 2024-04-02T11:26:27.531 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stderr:Suppressions used: 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stderr: count bytes template 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-02T11:26:27.532 INFO:teuthology.orchestra.run.smithi133.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:27.556 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T11:26:27.583 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:27.869 INFO:teuthology.orchestra.run.smithi133.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:27.870 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:27,871 seastar - Reactor backend: linux-aio 2024-04-02T11:26:27.875 INFO:teuthology.orchestra.run.smithi133.stderr:==29753==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:27.994 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:27,995 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-02T11:26:27.994 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:27,995 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-02T11:26:27.995 INFO:teuthology.orchestra.run.smithi133.stderr:DEBUG 2024-04-02 11:26:27,996 [shard 0:main] seastar - reactor::drain 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr:Suppressions used: 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr: count bytes template 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:28.524 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-02T11:26:28.583 INFO:teuthology.orchestra.run.smithi133.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:28.586 INFO:teuthology.orchestra.run.smithi133.stderr:INFO 2024-04-02 11:26:28,587 seastar - Reactor backend: linux-aio 2024-04-02T11:26:28.595 INFO:teuthology.orchestra.run.smithi133.stderr:==29751==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:32.047 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:Suppressions used: 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stderr: count bytes template 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stderr:----------------------------------------------------- 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-02T11:26:32.048 INFO:teuthology.orchestra.run.smithi133.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:32.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T11:26:32.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-02T11:26:32.129 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:32.130 INFO:tasks.ceph:role: osd.2 2024-04-02T11:26:32.130 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T11:26:32.130 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T11:26:32.208 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T11:26:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T11:26:32.209 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T11:26:32.703 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-02T11:26:32.705 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-02T11:26:32.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-02T11:26:32.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-02T11:26:32.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-02T11:26:32.939 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-04-02T11:26:32.939 INFO:tasks.ceph:role: osd.3 2024-04-02T11:26:32.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T11:26:32.940 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T11:26:33.016 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T11:26:33.016 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T11:26:33.016 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T11:26:33.016 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T11:26:33.017 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T11:26:33.473 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-02T11:26:33.475 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-02T11:26:33.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-04-02T11:26:33.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-02T11:26:33.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:33.934 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:33.935 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:33,936 seastar - Reactor backend: linux-aio 2024-04-02T11:26:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:==28816==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:34.074 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:34,074 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-02T11:26:34.074 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:34,074 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-04-02T11:26:34.074 INFO:teuthology.orchestra.run.smithi138.stderr:DEBUG 2024-04-02 11:26:34,075 [shard 0:main] seastar - reactor::drain 2024-04-02T11:26:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:34.648 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2024-04-02T11:26:34.648 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2024-04-02T11:26:34.648 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:34.648 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:34.648 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-02T11:26:34.703 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:34.706 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:34,707 seastar - Reactor backend: linux-aio 2024-04-02T11:26:34.715 INFO:teuthology.orchestra.run.smithi138.stderr:==28812==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:34.780 INFO:teuthology.orchestra.run.smithi138.stderr:Reactor stalled for 65 ms on shard 0. Backtrace: 0x6bddd 0xb9de8c9 0xb8b1390 0xb8b290c 0xb8b2b22 0xb8b2c78 0xb8b3141 0x54daf 0x172a9 0x174bb 0x2eab 0x4a09d 0x35b7c 0x3689e 0x36a1880 0x5dcdb72 0x5dce034 0x5dce416 0x5e2a3a4 0x5da8f20 0x5da9454 0x5bee4a7 0x5c0956b 0x6ee1bf4 0x6ef33c7 0x6ef4864 0x6ef48e9 0x6ef49eb 0x6ef4ae3 0x6ef4cf0 0x6ef4ed6 0x6ef5225 0x6ef55c9 0x6ef57f9 0x6ef5a63 0x6ef5d30 0x6ef5f4e 0x6ef6006 0x6ef60cc 0x6ef61ad 0x5a54aa8 0x5a54bb2 0x5a54c6e 0x5a54d2a 0x5a54de6 0x5a54f33 0x5a550be 0x5a5547f 0x5a71efc 0xb8c4015 0xb8de2af 0xb97f2ad 0xb9808fb 0xb65d063 0xb65d9df 0x368b58a 0x3feaf 0x3ff5f 0x3477444 2024-04-02T11:26:34.780 INFO:teuthology.orchestra.run.smithi138.stderr:kernel callstack: 2024-04-02T11:26:38.163 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-02T11:26:38.164 INFO:teuthology.orchestra.run.smithi138.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:38.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T11:26:38.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:38,503 seastar - Reactor backend: linux-aio 2024-04-02T11:26:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:==29625==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:38.626 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:38,627 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-04-02T11:26:38.626 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:38,627 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-04-02T11:26:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:DEBUG 2024-04-02 11:26:38,628 [shard 0:main] seastar - reactor::drain 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:39.157 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-02T11:26:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:WARNING: debug mode. Not for benchmarking or production 2024-04-02T11:26:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:INFO 2024-04-02 11:26:39,217 seastar - Reactor backend: linux-aio 2024-04-02T11:26:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:==29623==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-04-02T11:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:Suppressions used: 2024-04-02T11:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr: count bytes template 2024-04-02T11:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr: 1 8 MallocExtension::Register 2024-04-02T11:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:----------------------------------------------------- 2024-04-02T11:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-02T11:26:42.700 INFO:teuthology.orchestra.run.smithi138.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 0fac6a5d-b779-400c-9b2f-b14c76190269 2024-04-02T11:26:42.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T11:26:42.750 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-02T11:26:42.750 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:42.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-02T11:26:42.782 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:42.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-02T11:26:42.856 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:42.856 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-02T11:26:42.883 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:42.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-02T11:26:42.949 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:42.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-02T11:26:43.016 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:43.016 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-02T11:26:43.034 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:43.034 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-02T11:26:43.071 INFO:tasks.ceph:Adding keys to all mons... 2024-04-02T11:26:43.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBg6wtmGL6xDRAAo0qeLF5a1aeUd9Tqu6BrGw== 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBk6wtm4p4mLBAAq5g1J8rzNRvtvTr5LwU4Rg== 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBd6wtmpmI+FxAAHwrX5227D/SyjUb0eb+xKw== 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBq6wtmGqiOMxAASXiFIMP84e9l75C3Ytgf6g== 2024-04-02T11:26:43.102 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-04-02T11:26:43.103 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBv6wtmVsEiFhAAFvuk3LzI0/F6+qjo/vqefA== 2024-04-02T11:26:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-02T11:26:43.103 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBd6wtmULuCGhAAEALvm+hbzc6eTFVltakA5w== 2024-04-02T11:26:43.103 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-04-02T11:26:43.103 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBd6wtmgceCHhAAVsDbJIImtZkT5IuEwdo6iA== 2024-04-02T11:26:43.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T11:26:43.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T11:26:43.274 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T11:26:43.336 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T11:26:43.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T11:26:43.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T11:26:43.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T11:26:43.591 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-02T11:26:43.591 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-02T11:26:43.618 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T11:26:43.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.733+0000 7f36e3a35a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.733+0000 7f36e3a35a80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-02T11:26:43.774 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-02T11:26:43.828 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-02T11:26:43.829 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-02T11:26:43.829 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-02T11:26:43.871 INFO:tasks.ceph.mon.a:Started 2024-04-02T11:26:43.871 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-02T11:26:43.871 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-02T11:26:43.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-02T11:26:43.874 INFO:tasks.ceph.mgr.x:Started 2024-04-02T11:26:43.874 INFO:tasks.ceph:Setting crush tunables to default 2024-04-02T11:26:43.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-04-02T11:26:43.956 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-02T11:26:43.955+0000 7f941916ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.956 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-02T11:26:43.955+0000 7f941916ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.956 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-02T11:26:43.955+0000 7f941916ca80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.968 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:43.968+0000 7f3a3de32000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.979+0000 7f04e475d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.985+0000 7f04e475d640 1 Processor -- start 2024-04-02T11:26:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.985+0000 7f04e475d640 1 -- start start 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e475d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e475d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04dc09d9d0 con 0x7f04dc131920 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e475d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f04dc09d650 con 0x7f04dc09f2b0 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e24f9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e1cf8640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f04dc131920 0x7f04dc09a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:59834/0 (socket says 172.21.15.133:59834) 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e1cf8640 1 -- 172.21.15.133:0/448481854 learned_addr learned my addr 172.21.15.133:0/448481854 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e24f9640 1 -- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 msgr2=0x7f04dc1313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e24f9640 1 -- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 msgr2=0x7f04dc1313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e24f9640 1 --2- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T11:26:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.987+0000 7f04e24f9640 1 --2- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993243242 0 0) 0x7f04dc09d9d0 con 0x7f04dc131920 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04c8003580 con 0x7f04dc131920 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4037165178 0 0) 0x7f04c8003580 con 0x7f04dc131920 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 msgr2=0x7f04dc1313e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 --2- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc131140 con 0x7f04dc131920 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 3330554182 0 0) 0x7f04cc0015a0 con 0x7f04dc131920 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 >> v1:172.21.15.133:6789/0 conn(0x7f04dc131920 legacy=0x7f04dc09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:43.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 --2- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:43.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04c80044f0 con 0x7f04c80039d0 2024-04-02T11:26:43.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e24f9640 1 --2- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:43.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.989+0000 7f04e24f9640 1 -- 172.21.15.133:0/448481854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c8003580 con 0x7f04c80039d0 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.990+0000 7f04e24f9640 1 --2- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04c8003e20 crc :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36490cc312dfa5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f04d80095d0 con 0x7f04c80039d0 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f04d800c040 con 0x7f04c80039d0 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e14f7640 1 -- 172.21.15.133:0/448481854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f04d8009c40 con 0x7f04c80039d0 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 -- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 msgr2=0x7f04c8003e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 --2- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04c8003e20 crc :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 -- 172.21.15.133:0/448481854 shutdown_connections 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 --2- 172.21.15.133:0/448481854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04c8003e20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 --2- 172.21.15.133:0/448481854 >> v2:172.21.15.133:3300/0 conn(0x7f04dc09f2b0 0x7f04dc1313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.991+0000 7f04e475d640 1 -- 172.21.15.133:0/448481854 >> 172.21.15.133:0/448481854 conn(0x7f04dc0a03a0 msgr2=0x7f04dc0a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.992+0000 7f04e475d640 1 -- 172.21.15.133:0/448481854 shutdown_connections 2024-04-02T11:26:43.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.992+0000 7f04e475d640 1 -- 172.21.15.133:0/448481854 wait complete. 2024-04-02T11:26:43.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.992+0000 7f04e475d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:43.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.993+0000 7f04e475d640 1 Processor -- start 2024-04-02T11:26:43.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.993+0000 7f04e475d640 1 -- start start 2024-04-02T11:26:43.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.994+0000 7f04e475d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:43.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.994+0000 7f04e475d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc130c20 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.994+0000 7f04e24f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.994+0000 7f04e24f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56696/0 (socket says 172.21.15.133:56696) 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.994+0000 7f04e24f9640 1 -- 172.21.15.133:0/123255770 learned_addr learned my addr 172.21.15.133:0/123255770 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04e24f9640 1 -- 172.21.15.133:0/123255770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc123b30 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04e24f9640 1 --2- 172.21.15.133:0/123255770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f04d80037a0 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f04d8003900 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f04d8003ba0 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc122660 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc122f20 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f04d8003d50 con 0x7f04c80039d0 2024-04-02T11:26:43.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.995+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (crc 0 0 0) 0x7f04d8010b80 con 0x7f04c80039d0 2024-04-02T11:26:43.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.996+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc068e30 con 0x7f04c80039d0 2024-04-02T11:26:43.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:43.998+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f04d800f040 con 0x7f04c80039d0 2024-04-02T11:26:44.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.061+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f04dc09c0a0 con 0x7f04c80039d0 2024-04-02T11:26:44.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04d2ffd640 1 -- 172.21.15.133:0/123255770 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (crc 0 0 0) 0x7f04dc09c0a0 con 0x7f04c80039d0 2024-04-02T11:26:44.062 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 msgr2=0x7f04dc11faa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04e475d640 1 --2- 172.21.15.133:0/123255770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 shutdown_connections 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04e475d640 1 --2- 172.21.15.133:0/123255770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04c80039d0 0x7f04dc11faa0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.062+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 >> 172.21.15.133:0/123255770 conn(0x7f04dc0a03a0 msgr2=0x7f04dc0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.063+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 shutdown_connections 2024-04-02T11:26:44.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.063+0000 7f04e475d640 1 -- 172.21.15.133:0/123255770 wait complete. 2024-04-02T11:26:44.073 INFO:tasks.ceph:check_enable_crimson: True 2024-04-02T11:26:44.073 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-04-02T11:26:44.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-04-02T11:26:44.174 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:44.173+0000 7f3a3de32000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:44.174 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:44.174+0000 7f3a3de32000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:44.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.176+0000 7fe1c6a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:44.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.182+0000 7fe1c6a2d640 1 Processor -- start 2024-04-02T11:26:44.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.182+0000 7fe1c6a2d640 1 -- start start 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1c6a2d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1c0131780 0x7fe1c0131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1c6a2d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c00a6af0 con 0x7fe1c009edf0 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1c6a2d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c0073660 con 0x7fe1c0131780 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1c0131780 0x7fe1c0131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1c0131780 0x7fe1c0131b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56698/0 (socket says 172.21.15.133:56698) 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bf7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe1c009edf0 0x7fe1c009f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:59838/0 (socket says 172.21.15.133:59838) 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 -- 172.21.15.133:0/2247421169 learned_addr learned my addr 172.21.15.133:0/2247421169 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 -- 172.21.15.133:0/2247421169 >> v1:172.21.15.133:6789/0 conn(0x7fe1c009edf0 legacy=0x7fe1c009f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 -- 172.21.15.133:0/2247421169 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c00a67e0 con 0x7fe1c0131780 2024-04-02T11:26:44.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1beffd640 1 -- 172.21.15.133:0/2247421169 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374658152 0 0) 0x7fe1c00a6af0 con 0x7fe1c009edf0 2024-04-02T11:26:44.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1bffff640 1 --2- 172.21.15.133:0/2247421169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c0131780 0x7fe1c0131b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e627689331eb31b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:44.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1beffd640 1 -- 172.21.15.133:0/2247421169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1b0009e50 con 0x7fe1c0131780 2024-04-02T11:26:44.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1beffd640 1 -- 172.21.15.133:0/2247421169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe1b000e230 con 0x7fe1c0131780 2024-04-02T11:26:44.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.183+0000 7fe1beffd640 1 -- 172.21.15.133:0/2247421169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1b000e4d0 con 0x7fe1c0131780 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/2247421169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c0131780 msgr2=0x7fe1c0131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 --2- 172.21.15.133:0/2247421169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c0131780 0x7fe1c0131b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/2247421169 shutdown_connections 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 --2- 172.21.15.133:0/2247421169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c0131780 0x7fe1c0131b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/2247421169 >> 172.21.15.133:0/2247421169 conn(0x7fe1c00a4ba0 msgr2=0x7fe1c00a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/2247421169 shutdown_connections 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/2247421169 wait complete. 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.184+0000 7fe1c6a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.186+0000 7fe1c6a2d640 1 Processor -- start 2024-04-02T11:26:44.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.186+0000 7fe1c6a2d640 1 -- start start 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1c6a2d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1c6a2d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01313d0 con 0x7fe1c009edf0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56704/0 (socket says 172.21.15.133:56704) 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bffff640 1 -- 172.21.15.133:0/4196462234 learned_addr learned my addr 172.21.15.133:0/4196462234 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bffff640 1 -- 172.21.15.133:0/4196462234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c0120ff0 con 0x7fe1c009edf0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bffff640 1 --2- 172.21.15.133:0/4196462234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1b000ee50 con 0x7fe1c009edf0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe1b0015420 con 0x7fe1c009edf0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c0120a70 con 0x7fe1c009edf0 2024-04-02T11:26:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe1b00156c0 con 0x7fe1c009edf0 2024-04-02T11:26:44.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.187+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c011fa40 con 0x7fe1c009edf0 2024-04-02T11:26:44.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.188+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fe1b0015c00 con 0x7fe1c009edf0 2024-04-02T11:26:44.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.188+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (crc 0 0 0) 0x7fe1b0011380 con 0x7fe1c009edf0 2024-04-02T11:26:44.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.188+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c0120df0 con 0x7fe1c009edf0 2024-04-02T11:26:44.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.191+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fe1b00143b0 con 0x7fe1c009edf0 2024-04-02T11:26:44.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:44.263+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) -- 0x7fe1c0063400 con 0x7fe1c009edf0 2024-04-02T11:26:44.263 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-02T11:26:44.263+0000 7f9410ff4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-04-02T11:26:44.264 INFO:tasks.ceph.mon.a.smithi133.stderr:Please be aware that this feature is experimental, untested, 2024-04-02T11:26:44.264 INFO:tasks.ceph.mon.a.smithi133.stderr:unsupported, and may result in data corruption, data loss, 2024-04-02T11:26:44.264 INFO:tasks.ceph.mon.a.smithi133.stderr:and/or irreparable damage to your cluster. Do not use 2024-04-02T11:26:44.264 INFO:tasks.ceph.mon.a.smithi133.stderr:feature with important data. 2024-04-02T11:26:44.264 INFO:tasks.ceph.mon.a.smithi133.stderr: 2024-04-02T11:26:44.694 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:44.694+0000 7f3a3de32000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T11:26:44.746 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:44.746+0000 7f3a3de32000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T11:26:45.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.062+0000 7fe1bcff9640 1 -- 172.21.15.133:0/4196462234 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) ==== 99+0+0 (crc 0 0 0) 0x7fe1b00059f0 con 0x7fe1c009edf0 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 msgr2=0x7fe1c01201d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 --2- 172.21.15.133:0/4196462234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 shutdown_connections 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 --2- 172.21.15.133:0/4196462234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c009edf0 0x7fe1c01201d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 >> 172.21.15.133:0/4196462234 conn(0x7fe1c00a4ba0 msgr2=0x7fe1c01ba650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 shutdown_connections 2024-04-02T11:26:45.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:45.063+0000 7fe1c6a2d640 1 -- 172.21.15.133:0/4196462234 wait complete. 2024-04-02T11:26:45.075 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-02T11:26:45.076 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:45.076 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-02T11:26:45.103 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:26:45.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-02T11:26:45.170 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:45.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-02T11:26:45.197 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-02T11:26:45.197 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-02T11:26:45.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 98a4ef3a-6098-49fd-8bf6-b5d47db5b01c 0 2024-04-02T11:26:45.267 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.267+0000 7f3a3de32000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T11:26:45.324 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.324+0000 7f3a3de32000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T11:26:45.371 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.371+0000 7f3a3de32000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T11:26:45.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.370+0000 7f98ffd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.376+0000 7f98ffd9d640 1 Processor -- start 2024-04-02T11:26:45.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.376+0000 7f98ffd9d640 1 -- start start 2024-04-02T11:26:45.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98ffd9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98f811c170 0x7f98f811e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98ffd9d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f811ebc0 con 0x7f98f81136b0 2024-04-02T11:26:45.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98ffd9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f98f811fd80 con 0x7f98f811c170 2024-04-02T11:26:45.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98fd338640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98f811c170 0x7f98f811e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98fd338640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98f811c170 0x7f98f811e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35380/0 (socket says 172.21.15.138:35380) 2024-04-02T11:26:45.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98fd338640 1 -- 172.21.15.138:0/3431351128 learned_addr learned my addr 172.21.15.138:0/3431351128 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-02T11:26:45.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.377+0000 7f98fdb39640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f98f81136b0 0x7f98f8113ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.138:41670/0 (socket says 172.21.15.138:41670) 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fd338640 1 -- 172.21.15.138:0/3431351128 >> v1:172.21.15.133:6789/0 conn(0x7f98f81136b0 legacy=0x7f98f8113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fd338640 1 -- 172.21.15.138:0/3431351128 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f811fef0 con 0x7f98f811c170 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fd338640 1 --2- 172.21.15.138:0/3431351128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f811c170 0x7f98f811e590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a7c8383409943 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fcb37640 1 -- 172.21.15.138:0/3431351128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e8009e50 con 0x7f98f811c170 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fcb37640 1 -- 172.21.15.138:0/3431351128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f98e800e230 con 0x7f98f811c170 2024-04-02T11:26:45.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.378+0000 7f98fcb37640 1 -- 172.21.15.138:0/3431351128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e800e4d0 con 0x7f98f811c170 2024-04-02T11:26:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 -- 172.21.15.138:0/3431351128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f811c170 msgr2=0x7f98f811e590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 --2- 172.21.15.138:0/3431351128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f811c170 0x7f98f811e590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 -- 172.21.15.138:0/3431351128 shutdown_connections 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 --2- 172.21.15.138:0/3431351128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f811c170 0x7f98f811e590 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 -- 172.21.15.138:0/3431351128 >> 172.21.15.138:0/3431351128 conn(0x7f98f806e6c0 msgr2=0x7f98f806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 -- 172.21.15.138:0/3431351128 shutdown_connections 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 1 -- 172.21.15.138:0/3431351128 wait complete. 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.379+0000 7f98ffd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98ffd9d640 1 Processor -- start 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98ffd9d640 1 -- start start 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98ffd9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98ffd9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81ae2b0 con 0x7f98f81136b0 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98fdb39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98fdb39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35384/0 (socket says 172.21.15.138:35384) 2024-04-02T11:26:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.380+0000 7f98fdb39640 1 -- 172.21.15.138:0/137830004 learned_addr learned my addr 172.21.15.138:0/137830004 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-02T11:26:45.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98fdb39640 1 -- 172.21.15.138:0/137830004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f81ae3f0 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98fdb39640 1 --2- 172.21.15.138:0/137830004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98f40059a0 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f98f4005b00 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81ae640 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98f4005da0 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a8040 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f98f4007440 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.381+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 747+0+0 (crc 0 0 0) 0x7f98f400fa30 con 0x7f98f81136b0 2024-04-02T11:26:45.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.382+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f8063400 con 0x7f98f81136b0 2024-04-02T11:26:45.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.385+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f98f400fc90 con 0x7f98f81136b0 2024-04-02T11:26:45.426 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.426+0000 7f3a3de32000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T11:26:45.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.457+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "98a4ef3a-6098-49fd-8bf6-b5d47db5b01c", "id": 0} v 0) -- 0x7f98f811c210 con 0x7f98f81136b0 2024-04-02T11:26:45.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.458+0000 7f98ee7fc640 1 -- 172.21.15.138:0/137830004 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98a4ef3a-6098-49fd-8bf6-b5d47db5b01c", "id": 0}]=0 v4) ==== 112+0+2 (crc 0 0 0) 0x7f98f400fe30 con 0x7f98f81136b0 2024-04-02T11:26:45.458 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 msgr2=0x7f98f81adce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 --2- 172.21.15.138:0/137830004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 shutdown_connections 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 --2- 172.21.15.138:0/137830004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f81136b0 0x7f98f81adce0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 >> 172.21.15.138:0/137830004 conn(0x7f98f806e6c0 msgr2=0x7f98f810fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 shutdown_connections 2024-04-02T11:26:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.459+0000 7f98ffd9d640 1 -- 172.21.15.138:0/137830004 wait complete. 2024-04-02T11:26:45.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new a7ca277f-c9c9-4571-9fcd-b183ed099431 1 2024-04-02T11:26:45.471 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.470+0000 7f3a3de32000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T11:26:45.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.574+0000 7f5030f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.580+0000 7f5030f1d640 1 Processor -- start 2024-04-02T11:26:45.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.580+0000 7f5030f1d640 1 -- start start 2024-04-02T11:26:45.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f5030f1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f502c1136b0 0x7f502c113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f5030f1d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f502c118b20 con 0x7f502c118000 2024-04-02T11:26:45.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f5030f1d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f502c118cb0 con 0x7f502c1136b0 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f502a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f502c1136b0 0x7f502c113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f5023fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f502c118000 0x7f502c118410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.138:41680/0 (socket says 172.21.15.138:41680) 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f502a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f502c1136b0 0x7f502c113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35400/0 (socket says 172.21.15.138:35400) 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f5023fff640 1 -- 172.21.15.138:0/3862958075 learned_addr learned my addr 172.21.15.138:0/3862958075 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f502a59c640 1 -- 172.21.15.138:0/3862958075 >> v1:172.21.15.133:6789/0 conn(0x7f502c118000 legacy=0x7f502c118410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.581+0000 7f502a59c640 1 -- 172.21.15.138:0/3862958075 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c118e20 con 0x7f502c1136b0 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f502a59c640 1 --2- 172.21.15.138:0/3862958075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c113ac0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5472e479342194d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f5029d9b640 1 -- 172.21.15.138:0/3862958075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f501c009d90 con 0x7f502c1136b0 2024-04-02T11:26:45.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f5029d9b640 1 -- 172.21.15.138:0/3862958075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f501c00e230 con 0x7f502c1136b0 2024-04-02T11:26:45.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f5029d9b640 1 -- 172.21.15.138:0/3862958075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f501c00e4d0 con 0x7f502c1136b0 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f5030f1d640 1 -- 172.21.15.138:0/3862958075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 msgr2=0x7f502c113ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.582+0000 7f5030f1d640 1 --2- 172.21.15.138:0/3862958075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c113ac0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 1 -- 172.21.15.138:0/3862958075 shutdown_connections 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 1 --2- 172.21.15.138:0/3862958075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c113ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 1 -- 172.21.15.138:0/3862958075 >> 172.21.15.138:0/3862958075 conn(0x7f502c06e6c0 msgr2=0x7f502c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 1 -- 172.21.15.138:0/3862958075 shutdown_connections 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 1 -- 172.21.15.138:0/3862958075 wait complete. 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.583+0000 7f5030f1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f5030f1d640 1 Processor -- start 2024-04-02T11:26:45.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f5030f1d640 1 -- start start 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f5030f1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f5030f1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f502c1ae280 con 0x7f502c1136b0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f502a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f502a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35414/0 (socket says 172.21.15.138:35414) 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f502a59c640 1 -- 172.21.15.138:0/1855123174 learned_addr learned my addr 172.21.15.138:0/1855123174 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.584+0000 7f502a59c640 1 -- 172.21.15.138:0/1855123174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c1ae3c0 con 0x7f502c1136b0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f502a59c640 1 --2- 172.21.15.138:0/1855123174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f501c014750 con 0x7f502c1136b0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f501c0148b0 con 0x7f502c1136b0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c1ae5b0 con 0x7f502c1136b0 2024-04-02T11:26:45.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f501c014b50 con 0x7f502c1136b0 2024-04-02T11:26:45.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c1a8010 con 0x7f502c1136b0 2024-04-02T11:26:45.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f501c013790 con 0x7f502c1136b0 2024-04-02T11:26:45.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.585+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 862+0+0 (crc 0 0 0) 0x7f501c019040 con 0x7f502c1136b0 2024-04-02T11:26:45.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.586+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c1ae7a0 con 0x7f502c1136b0 2024-04-02T11:26:45.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.588+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f501c014cb0 con 0x7f502c1136b0 2024-04-02T11:26:45.594 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.594+0000 7f3a3de32000 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T11:26:45.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.661+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7ca277f-c9c9-4571-9fcd-b183ed099431", "id": 1} v 0) -- 0x7f502c118240 con 0x7f502c1136b0 2024-04-02T11:26:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.662+0000 7f5022ffd640 1 -- 172.21.15.138:0/1855123174 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7ca277f-c9c9-4571-9fcd-b183ed099431", "id": 1}]=0 v5) ==== 112+0+2 (crc 0 0 0) 0x7f501c014e50 con 0x7f502c1136b0 2024-04-02T11:26:45.663 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 msgr2=0x7f502c1adcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 --2- 172.21.15.138:0/1855123174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 shutdown_connections 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 --2- 172.21.15.138:0/1855123174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c1136b0 0x7f502c1adcb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 >> 172.21.15.138:0/1855123174 conn(0x7f502c06e6c0 msgr2=0x7f502c116b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 shutdown_connections 2024-04-02T11:26:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.663+0000 7f5030f1d640 1 -- 172.21.15.138:0/1855123174 wait complete. 2024-04-02T11:26:45.665 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.665+0000 7f3a3de32000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T11:26:45.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new d4688372-a095-4687-9a15-67e86164a07c 2 2024-04-02T11:26:45.714 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.714+0000 7f3a3de32000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T11:26:45.768 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.768+0000 7f3a3de32000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T11:26:45.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.776+0000 7f9e2cbc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.782+0000 7f9e2cbc6640 1 Processor -- start 2024-04-02T11:26:45.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.782+0000 7f9e2cbc6640 1 -- start start 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2cbc6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e281136b0 0x7f9e28113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2cbc6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e28118b20 con 0x7f9e28118000 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2cbc6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f9e28118cb0 con 0x7f9e281136b0 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e281136b0 0x7f9e28113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e25d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9e28118000 0x7f9e28118410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.138:41688/0 (socket says 172.21.15.138:41688) 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e25d9b640 1 -- 172.21.15.138:0/2085196633 learned_addr learned my addr 172.21.15.138:0/2085196633 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2659c640 1 -- 172.21.15.138:0/2085196633 >> v1:172.21.15.133:6789/0 conn(0x7f9e28118000 legacy=0x7f9e28118410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2559a640 1 -- 172.21.15.138:0/2085196633 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122982790 0 0) 0x7f9e28118b20 con 0x7f9e28118000 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.783+0000 7f9e2659c640 1 -- 172.21.15.138:0/2085196633 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e28118e20 con 0x7f9e281136b0 2024-04-02T11:26:45.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.784+0000 7f9e2659c640 1 --2- 172.21.15.138:0/2085196633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e28113ac0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d641123d652b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.784+0000 7f9e2559a640 1 -- 172.21.15.138:0/2085196633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9e18009d90 con 0x7f9e281136b0 2024-04-02T11:26:45.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.784+0000 7f9e2559a640 1 -- 172.21.15.138:0/2085196633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9e1800e230 con 0x7f9e281136b0 2024-04-02T11:26:45.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.784+0000 7f9e2559a640 1 -- 172.21.15.138:0/2085196633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9e1800e4d0 con 0x7f9e281136b0 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2085196633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 msgr2=0x7f9e28113ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 --2- 172.21.15.138:0/2085196633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e28113ac0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2085196633 shutdown_connections 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 --2- 172.21.15.138:0/2085196633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e28113ac0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2085196633 >> 172.21.15.138:0/2085196633 conn(0x7f9e2806e6c0 msgr2=0x7f9e2806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2085196633 shutdown_connections 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2085196633 wait complete. 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.785+0000 7f9e2cbc6640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2cbc6640 1 Processor -- start 2024-04-02T11:26:45.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2cbc6640 1 -- start start 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2cbc6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2cbc6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e281ae060 con 0x7f9e281136b0 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35440/0 (socket says 172.21.15.138:35440) 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.786+0000 7f9e2659c640 1 -- 172.21.15.138:0/2591481928 learned_addr learned my addr 172.21.15.138:0/2591481928 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e2659c640 1 -- 172.21.15.138:0/2591481928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e281ae1a0 con 0x7f9e281136b0 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e2659c640 1 --2- 172.21.15.138:0/2591481928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:45.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9e18013740 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9e180138a0 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e281ae390 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9e18013b40 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e281a7df0 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f9e18012780 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.787+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 977+0+0 (crc 0 0 0) 0x7f9e18018040 con 0x7f9e281136b0 2024-04-02T11:26:45.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.788+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e281ae580 con 0x7f9e281136b0 2024-04-02T11:26:45.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.790+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f9e18013cf0 con 0x7f9e281136b0 2024-04-02T11:26:45.817 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:45.817+0000 7f3a3de32000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T11:26:45.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.863+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4688372-a095-4687-9a15-67e86164a07c", "id": 2} v 0) -- 0x7f9e28118140 con 0x7f9e281136b0 2024-04-02T11:26:45.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.864+0000 7f9e16ffd640 1 -- 172.21.15.138:0/2591481928 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4688372-a095-4687-9a15-67e86164a07c", "id": 2}]=0 v6) ==== 112+0+2 (crc 0 0 0) 0x7f9e18012c10 con 0x7f9e281136b0 2024-04-02T11:26:45.864 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 msgr2=0x7f9e281ada90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 --2- 172.21.15.138:0/2591481928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 shutdown_connections 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 --2- 172.21.15.138:0/2591481928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e281136b0 0x7f9e281ada90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 >> 172.21.15.138:0/2591481928 conn(0x7f9e2806e6c0 msgr2=0x7f9e28115110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 shutdown_connections 2024-04-02T11:26:45.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:45.865+0000 7f9e2cbc6640 1 -- 172.21.15.138:0/2591481928 wait complete. 2024-04-02T11:26:45.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 514ade77-a0ef-4940-84f3-2a75ebe04025 3 2024-04-02T11:26:46.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.019+0000 7fbb0082d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:46.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.025+0000 7fbb0082d640 1 Processor -- start 2024-04-02T11:26:46.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.025+0000 7fbb0082d640 1 -- start start 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbb0082d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbafc11c170 0x7fbafc11e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbb0082d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc11ebc0 con 0x7fbafc1136b0 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbb0082d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc11fd80 con 0x7fbafc11c170 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbaf9d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbafc11c170 0x7fbafc11e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbafa59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbafc1136b0 0x7fbafc113ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.138:41702/0 (socket says 172.21.15.138:41702) 2024-04-02T11:26:46.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbafa59c640 1 -- 172.21.15.138:0/1665830837 learned_addr learned my addr 172.21.15.138:0/1665830837 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbaf959a640 1 -- 172.21.15.138:0/1665830837 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613715476 0 0) 0x7fbafc11ebc0 con 0x7fbafc1136b0 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbaf959a640 1 -- 172.21.15.138:0/1665830837 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbae4003580 con 0x7fbafc1136b0 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbaf9d9b640 1 -- 172.21.15.138:0/1665830837 >> v1:172.21.15.133:6789/0 conn(0x7fbafc1136b0 legacy=0x7fbafc113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.026+0000 7fbaf9d9b640 1 -- 172.21.15.138:0/1665830837 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc11fef0 con 0x7fbafc11c170 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbaf9d9b640 1 --2- 172.21.15.138:0/1665830837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc11c170 0x7fbafc11e590 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be51edaaf050883 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbaf959a640 1 -- 172.21.15.138:0/1665830837 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbaec009d90 con 0x7fbafc11c170 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbaf959a640 1 -- 172.21.15.138:0/1665830837 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbaec00e230 con 0x7fbafc11c170 2024-04-02T11:26:46.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbaf959a640 1 -- 172.21.15.138:0/1665830837 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbaec00e4d0 con 0x7fbafc11c170 2024-04-02T11:26:46.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbb0082d640 1 -- 172.21.15.138:0/1665830837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc11c170 msgr2=0x7fbafc11e590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.027+0000 7fbb0082d640 1 --2- 172.21.15.138:0/1665830837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc11c170 0x7fbafc11e590 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 1 -- 172.21.15.138:0/1665830837 shutdown_connections 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 1 --2- 172.21.15.138:0/1665830837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc11c170 0x7fbafc11e590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 1 -- 172.21.15.138:0/1665830837 >> 172.21.15.138:0/1665830837 conn(0x7fbafc06e6c0 msgr2=0x7fbafc06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 1 -- 172.21.15.138:0/1665830837 shutdown_connections 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 1 -- 172.21.15.138:0/1665830837 wait complete. 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.028+0000 7fbb0082d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbb0082d640 1 Processor -- start 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbb0082d640 1 -- start start 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbb0082d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbb0082d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc1ae0b0 con 0x7fbafc1136b0 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbafa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbafa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.138:35452/0 (socket says 172.21.15.138:35452) 2024-04-02T11:26:46.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.029+0000 7fbafa59c640 1 -- 172.21.15.138:0/1198728719 learned_addr learned my addr 172.21.15.138:0/1198728719 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbafa59c640 1 -- 172.21.15.138:0/1198728719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc1ae1f0 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbafa59c640 1 --2- 172.21.15.138:0/1198728719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbaf0005a60 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbaf0005bc0 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc1ae440 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbaf0005e60 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1a7e40 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fbaf000cbd0 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.030+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1092+0+0 (crc 0 0 0) 0x7fbaf000fb90 con 0x7fbafc1136b0 2024-04-02T11:26:46.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.031+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc1ae630 con 0x7fbafc1136b0 2024-04-02T11:26:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.034+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fbaf000fda0 con 0x7fbafc1136b0 2024-04-02T11:26:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.104+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "514ade77-a0ef-4940-84f3-2a75ebe04025", "id": 3} v 0) -- 0x7fbafc11c210 con 0x7fbafc1136b0 2024-04-02T11:26:46.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.108+0000 7fbaeaffd640 1 -- 172.21.15.138:0/1198728719 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "514ade77-a0ef-4940-84f3-2a75ebe04025", "id": 3}]=0 v7) ==== 112+0+2 (crc 0 0 0) 0x7fbaf000f6e0 con 0x7fbafc1136b0 2024-04-02T11:26:46.109 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 msgr2=0x7fbafc1adae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 --2- 172.21.15.138:0/1198728719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 shutdown_connections 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 --2- 172.21.15.138:0/1198728719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbafc1136b0 0x7fbafc1adae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 >> 172.21.15.138:0/1198728719 conn(0x7fbafc06e6c0 msgr2=0x7fbafc111230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.109+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 shutdown_connections 2024-04-02T11:26:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-02T11:26:46.110+0000 7fbb0082d640 1 -- 172.21.15.138:0/1198728719 wait complete. 2024-04-02T11:26:46.120 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-04-02T11:26:46.153 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-02T11:26:46.153 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-04-02T11:26:46.195 INFO:tasks.ceph.osd.0:Started 2024-04-02T11:26:46.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-04-02T11:26:46.207 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.206+0000 7f3a3de32000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T11:26:46.227 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-02T11:26:46.227 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-04-02T11:26:46.270 INFO:tasks.ceph.osd.1:Started 2024-04-02T11:26:46.270 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-04-02T11:26:46.302 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-02T11:26:46.302 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-04-02T11:26:46.334 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.334+0000 7f3a3de32000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T11:26:46.336 INFO:tasks.ceph.osd.2:Started 2024-04-02T11:26:46.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-04-02T11:26:46.367 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-02T11:26:46.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-04-02T11:26:46.409 INFO:tasks.ceph.osd.3:Started 2024-04-02T11:26:46.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T11:26:46.463 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.463+0000 7f3a3de32000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T11:26:46.515 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.515+0000 7f3a3de32000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T11:26:46.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.523+0000 7f6b6066d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:46.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.528+0000 7f6b6066d640 1 Processor -- start 2024-04-02T11:26:46.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.528+0000 7f6b6066d640 1 -- start start 2024-04-02T11:26:46.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b6066d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b58131780 0x7f6b58131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b6066d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b580a6af0 con 0x7f6b5809edf0 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b6066d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f6b58073660 con 0x7f6b58131780 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b5e409640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b58131780 0x7f6b58131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b5dc08640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6b5809edf0 0x7f6b5809f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54444/0 (socket says 172.21.15.133:54444) 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.529+0000 7f6b5dc08640 1 -- 172.21.15.133:0/3124639950 learned_addr learned my addr 172.21.15.133:0/3124639950 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5e409640 1 -- 172.21.15.133:0/3124639950 >> v1:172.21.15.133:6789/0 conn(0x7f6b5809edf0 legacy=0x7f6b5809f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5e409640 1 -- 172.21.15.133:0/3124639950 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b580a67e0 con 0x7f6b58131780 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5d407640 1 -- 172.21.15.133:0/3124639950 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861106914 0 0) 0x7f6b580a6af0 con 0x7f6b5809edf0 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5e409640 1 --2- 172.21.15.133:0/3124639950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b58131780 0x7f6b58131b90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50ebd07a100e5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5d407640 1 -- 172.21.15.133:0/3124639950 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6b50009d90 con 0x7f6b58131780 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5d407640 1 -- 172.21.15.133:0/3124639950 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6b5000e230 con 0x7f6b58131780 2024-04-02T11:26:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.530+0000 7f6b5d407640 1 -- 172.21.15.133:0/3124639950 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6b5000e4d0 con 0x7f6b58131780 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 -- 172.21.15.133:0/3124639950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b58131780 msgr2=0x7f6b58131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 --2- 172.21.15.133:0/3124639950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b58131780 0x7f6b58131b90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 -- 172.21.15.133:0/3124639950 shutdown_connections 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 --2- 172.21.15.133:0/3124639950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b58131780 0x7f6b58131b90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 -- 172.21.15.133:0/3124639950 >> 172.21.15.133:0/3124639950 conn(0x7f6b580a4ba0 msgr2=0x7f6b580a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 -- 172.21.15.133:0/3124639950 shutdown_connections 2024-04-02T11:26:46.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 1 -- 172.21.15.133:0/3124639950 wait complete. 2024-04-02T11:26:46.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.531+0000 7f6b6066d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:46.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.532+0000 7f6b6066d640 1 Processor -- start 2024-04-02T11:26:46.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.532+0000 7f6b6066d640 1 -- start start 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.533+0000 7f6b6066d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.533+0000 7f6b6066d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b581204d0 con 0x7f6b5809edf0 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b5e409640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b5e409640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46758/0 (socket says 172.21.15.133:46758) 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b5e409640 1 -- 172.21.15.133:0/2449851981 learned_addr learned my addr 172.21.15.133:0/2449851981 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b5e409640 1 -- 172.21.15.133:0/2449851981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b58120760 con 0x7f6b5809edf0 2024-04-02T11:26:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b5e409640 1 --2- 172.21.15.133:0/2449851981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:46.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6b50014620 con 0x7f6b5809edf0 2024-04-02T11:26:46.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6b50014780 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b58121460 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6b50014a20 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b58121fc0 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f6b50013400 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.534+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7f6b50019040 con 0x7f6b5809edf0 2024-04-02T11:26:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.535+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5811f490 con 0x7f6b5809edf0 2024-04-02T11:26:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.537+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7f6b50011c60 con 0x7f6b5809edf0 2024-04-02T11:26:46.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.601+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b58063400 con 0x7f6b5809edf0 2024-04-02T11:26:46.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.602+0000 7f6b56ffd640 1 -- 172.21.15.133:0/2449851981 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7f6b500116a0 con 0x7f6b5809edf0 2024-04-02T11:26:46.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:46.603 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:26:46.106374+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T11:26:46.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 msgr2=0x7f6b5811df60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 --2- 172.21.15.133:0/2449851981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 shutdown_connections 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 --2- 172.21.15.133:0/2449851981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5809edf0 0x7f6b5811df60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 >> 172.21.15.133:0/2449851981 conn(0x7f6b580a4ba0 msgr2=0x7f6b58133470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 shutdown_connections 2024-04-02T11:26:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:46.603+0000 7f6b6066d640 1 -- 172.21.15.133:0/2449851981 wait complete. 2024-04-02T11:26:46.614 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-02T11:26:46.614 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-02T11:26:46.641 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.641+0000 7f3a3de32000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T11:26:46.742 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.742+0000 7f3a3de32000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T11:26:46.906 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.905+0000 7f3a3de32000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T11:26:46.951 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:46.950+0000 7f3a3de32000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T11:26:47.017 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T11:26:47.121 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:47.120+0000 7f3a3de32000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T11:26:47.121 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.117+0000 7fe2dca6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:47.122 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.122+0000 7fe2dca6d640 1 Processor -- start 2024-04-02T11:26:47.122 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.122+0000 7fe2dca6d640 1 -- start start 2024-04-02T11:26:47.124 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2dca6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2d809f910 0x7fe2d8139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2dca6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d80a69e0 con 0x7fe2d809edf0 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2dca6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d80a0070 con 0x7fe2d809f910 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2d5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2d809f910 0x7fe2d8139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2d659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe2d809edf0 0x7fe2d809f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54450/0 (socket says 172.21.15.133:54450) 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.123+0000 7fe2d659c640 1 -- 172.21.15.133:0/1981854992 learned_addr learned my addr 172.21.15.133:0/1981854992 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d5d9b640 1 -- 172.21.15.133:0/1981854992 >> v1:172.21.15.133:6789/0 conn(0x7fe2d809edf0 legacy=0x7fe2d809f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d5d9b640 1 -- 172.21.15.133:0/1981854992 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d80a66d0 con 0x7fe2d809f910 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d5d9b640 1 --2- 172.21.15.133:0/1981854992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809f910 0x7fe2d8139ad0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d18096815c54c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d559a640 1 -- 172.21.15.133:0/1981854992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe2c8009d90 con 0x7fe2d809f910 2024-04-02T11:26:47.125 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d559a640 1 -- 172.21.15.133:0/1981854992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe2c800e230 con 0x7fe2d809f910 2024-04-02T11:26:47.126 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.124+0000 7fe2d559a640 1 -- 172.21.15.133:0/1981854992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe2c800e4d0 con 0x7fe2d809f910 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 -- 172.21.15.133:0/1981854992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809f910 msgr2=0x7fe2d8139ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 --2- 172.21.15.133:0/1981854992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809f910 0x7fe2d8139ad0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 -- 172.21.15.133:0/1981854992 shutdown_connections 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 --2- 172.21.15.133:0/1981854992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809f910 0x7fe2d8139ad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 -- 172.21.15.133:0/1981854992 >> 172.21.15.133:0/1981854992 conn(0x7fe2d80a4a90 msgr2=0x7fe2d80a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 -- 172.21.15.133:0/1981854992 shutdown_connections 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 1 -- 172.21.15.133:0/1981854992 wait complete. 2024-04-02T11:26:47.127 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.125+0000 7fe2dca6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:47.128 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.127+0000 7fe2dca6d640 1 Processor -- start 2024-04-02T11:26:47.128 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.127+0000 7fe2dca6d640 1 -- start start 2024-04-02T11:26:47.130 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2dca6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:47.130 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2dca6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d811d280 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46784/0 (socket says 172.21.15.133:46784) 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2d659c640 1 -- 172.21.15.133:0/3236642332 learned_addr learned my addr 172.21.15.133:0/3236642332 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.128+0000 7fe2d659c640 1 -- 172.21.15.133:0/3236642332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d80718f0 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2d659c640 1 --2- 172.21.15.133:0/3236642332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe2cc005a60 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe2cc005bc0 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe2cc005e60 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d8071cf0 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d811e790 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fe2cc00cb10 con 0x7fe2d809edf0 2024-04-02T11:26:47.131 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.129+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7fe2cc00fb60 con 0x7fe2d809edf0 2024-04-02T11:26:47.132 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.130+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d8071a80 con 0x7fe2d809edf0 2024-04-02T11:26:47.132 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.132+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (crc 0 0 0) 0x7fe2cc00fdc0 con 0x7fe2d809edf0 2024-04-02T11:26:47.194 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.193+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2d8063400 con 0x7fe2d809edf0 2024-04-02T11:26:47.194 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.194+0000 7fe2c6ffd640 1 -- 172.21.15.133:0/3236642332 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7fe2cc00f640 con 0x7fe2d809edf0 2024-04-02T11:26:47.195 INFO:teuthology.misc.health.smithi133.stdout: 2024-04-02T11:26:47.195 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":7,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:26:46.106374+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 msgr2=0x7fe2d811d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 --2- 172.21.15.133:0/3236642332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 shutdown_connections 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 --2- 172.21.15.133:0/3236642332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2d809edf0 0x7fe2d811d700 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 >> 172.21.15.133:0/3236642332 conn(0x7fe2d80a4a90 msgr2=0x7fe2d80a12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 shutdown_connections 2024-04-02T11:26:47.196 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:47.195+0000 7fe2dca6d640 1 -- 172.21.15.133:0/3236642332 wait complete. 2024-04-02T11:26:47.205 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-02T11:26:48.005 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:48.004+0000 7f3a3de32000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T11:26:48.379 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:48.378+0000 7f3a3de32000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T11:26:48.436 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:48.435+0000 7f3a3de32000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T11:26:48.561 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:48.561+0000 7f3a3de32000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T11:26:49.573 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-02T11:26:49.573+0000 7f39dcc3d640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-02T11:26:53.609 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T11:26:53.708 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.705+0000 7fd8d306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:53.709 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.709+0000 7fd8d306d640 1 Processor -- start 2024-04-02T11:26:53.709 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.709+0000 7fd8d306d640 1 -- start start 2024-04-02T11:26:53.712 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8d306d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8cc070710 0x7fd8cc12f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:53.712 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8d306d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8cc09c410 con 0x7fd8cc12f930 2024-04-02T11:26:53.712 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8d306d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fd8cc070f50 con 0x7fd8cc070710 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8d0e09640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8cc070710 0x7fd8cc12f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8cbfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd8cc12f930 0x7fd8cc132430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54456/0 (socket says 172.21.15.133:54456) 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8d0e09640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8cc070710 0x7fd8cc12f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46800/0 (socket says 172.21.15.133:46800) 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.710+0000 7fd8cbfff640 1 -- 172.21.15.133:0/3257933621 learned_addr learned my addr 172.21.15.133:0/3257933621 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8d0e09640 1 -- 172.21.15.133:0/3257933621 >> v1:172.21.15.133:6789/0 conn(0x7fd8cc12f930 legacy=0x7fd8cc132430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8d0e09640 1 -- 172.21.15.133:0/3257933621 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc070b20 con 0x7fd8cc070710 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8d0e09640 1 --2- 172.21.15.133:0/3257933621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc12f3f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16299b48b27d967d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8cb7fe640 1 -- 172.21.15.133:0/3257933621 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd8bc009d90 con 0x7fd8cc070710 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8cb7fe640 1 -- 172.21.15.133:0/3257933621 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd8bc00e230 con 0x7fd8cc070710 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.711+0000 7fd8cb7fe640 1 -- 172.21.15.133:0/3257933621 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd8bc00e4d0 con 0x7fd8cc070710 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 -- 172.21.15.133:0/3257933621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 msgr2=0x7fd8cc12f3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 --2- 172.21.15.133:0/3257933621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc12f3f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.713 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 -- 172.21.15.133:0/3257933621 shutdown_connections 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 --2- 172.21.15.133:0/3257933621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc12f3f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 -- 172.21.15.133:0/3257933621 >> 172.21.15.133:0/3257933621 conn(0x7fd8cc09d180 msgr2=0x7fd8cc09d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 -- 172.21.15.133:0/3257933621 shutdown_connections 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 1 -- 172.21.15.133:0/3257933621 wait complete. 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.712+0000 7fd8d306d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.713+0000 7fd8d306d640 1 Processor -- start 2024-04-02T11:26:53.714 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.713+0000 7fd8d306d640 1 -- start start 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.714+0000 7fd8d306d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.714+0000 7fd8d306d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc12de70 con 0x7fd8cc070710 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d0e09640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d0e09640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46804/0 (socket says 172.21.15.133:46804) 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d0e09640 1 -- 172.21.15.133:0/415734526 learned_addr learned my addr 172.21.15.133:0/415734526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d0e09640 1 -- 172.21.15.133:0/415734526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc11b630 con 0x7fd8cc070710 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d0e09640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd8bc014620 con 0x7fd8cc070710 2024-04-02T11:26:53.716 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc11db20 con 0x7fd8cc070710 2024-04-02T11:26:53.717 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd8bc014780 con 0x7fd8cc070710 2024-04-02T11:26:53.717 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd8bc014a20 con 0x7fd8cc070710 2024-04-02T11:26:53.717 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.715+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc11d6a0 con 0x7fd8cc070710 2024-04-02T11:26:53.718 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.716+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 45778+0+0 (crc 0 0 0) 0x7fd8bc014c20 con 0x7fd8cc070710 2024-04-02T11:26:53.718 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.716+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc068e30 con 0x7fd8cc070710 2024-04-02T11:26:53.719 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.716+0000 7fd8c97fa640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 0x7fd8b803adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:53.719 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.716+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2319+0+0 (crc 0 0 0) 0x7fd8bc019040 con 0x7fd8cc070710 2024-04-02T11:26:53.719 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.717+0000 7fd8cbfff640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 0x7fd8b803adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:53.719 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.717+0000 7fd8cbfff640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 0x7fd8b803adb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:53.722 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.723+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fd8bc0185a0 con 0x7fd8cc070710 2024-04-02T11:26:53.900 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.900+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8cc0a22b0 con 0x7fd8cc070710 2024-04-02T11:26:53.901 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.900+0000 7fd8c97fa640 1 -- 172.21.15.133:0/415734526 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+6954 (crc 0 0 0) 0x7fd8cc0a22b0 con 0x7fd8cc070710 2024-04-02T11:26:53.901 INFO:teuthology.misc.health.smithi133.stdout: 2024-04-02T11:26:53.901 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":12,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:26:53.092326+0000","last_up_change":"2024-04-02T11:26:50.078833+0000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T11:26:50.584783+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6801","nonce":1793372331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1793372331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1793372331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":1793372331}]},"public_addr":"172.21.15.133:6801/1793372331","cluster_addr":"172.21.15.133:6802/1793372331","heartbeat_back_addr":"172.21.15.133:6806/1793372331","heartbeat_front_addr":"172.21.15.133:6807/1793372331","state":["exists","up"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":4121831248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4121831248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":4121831248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6813","nonce":4121831248}]},"public_addr":"172.21.15.133:6803/4121831248","cluster_addr":"172.21.15.133:6800/4121831248","heartbeat_back_addr":"172.21.15.133:6814/4121831248","heartbeat_front_addr":"172.21.15.133:6813/4121831248","state":["exists","up"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":1859851880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1859851880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":1859851880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1859851880}]},"public_addr":"172.21.15.138:6805/1859851880","cluster_addr":"172.21.15.138:6804/1859851880","heartbeat_back_addr":"172.21.15.138:6811/1859851880","heartbeat_front_addr":"172.21.15.138:6812/1859851880","state":["exists","up"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3056026735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3056026735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3056026735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":3056026735}]},"public_addr":"172.21.15.138:6806/3056026735","cluster_addr":"172.21.15.138:6807/3056026735","heartbeat_back_addr":"172.21.15.138:6808/3056026735","heartbeat_front_addr":"172.21.15.138:6809/3056026735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 msgr2=0x7fd8b803adb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 0x7fd8b803adb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 msgr2=0x7fd8cc119b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 shutdown_connections 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd8b80388a0 0x7fd8b803adb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 --2- 172.21.15.133:0/415734526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8cc070710 0x7fd8cc119b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 >> 172.21.15.133:0/415734526 conn(0x7fd8cc09d180 msgr2=0x7fd8cc0a13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 shutdown_connections 2024-04-02T11:26:53.904 INFO:teuthology.misc.health.smithi133.stderr:2024-04-02T11:26:53.903+0000 7fd8d306d640 1 -- 172.21.15.133:0/415734526 wait complete. 2024-04-02T11:26:53.913 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-02T11:26:53.913 INFO:tasks.ceph:Creating RBD pool 2024-04-02T11:26:53.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-02T11:26:54.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.017+0000 7f809d0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.023+0000 7f809d0ed640 1 Processor -- start 2024-04-02T11:26:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.023+0000 7f809d0ed640 1 -- start start 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.024+0000 7f809d0ed640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f809809f910 0x7f8098139b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809d0ed640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80980a0070 con 0x7f809809edf0 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809d0ed640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f80980a03f0 con 0x7f809809f910 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f809809f910 0x7f8098139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8096d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f809809edf0 0x7f809809f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54466/0 (socket says 172.21.15.133:54466) 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8096d9d640 1 -- 172.21.15.133:0/2139847585 learned_addr learned my addr 172.21.15.133:0/2139847585 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809659c640 1 -- 172.21.15.133:0/2139847585 >> v1:172.21.15.133:6789/0 conn(0x7f809809edf0 legacy=0x7f809809f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809659c640 1 -- 172.21.15.133:0/2139847585 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809806e8a0 con 0x7f809809f910 2024-04-02T11:26:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8095d9b640 1 -- 172.21.15.133:0/2139847585 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2122467176 0 0) 0x7f80980a0070 con 0x7f809809edf0 2024-04-02T11:26:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f809659c640 1 --2- 172.21.15.133:0/2139847585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809f910 0x7f8098139b70 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3403bbf4f2436c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8095d9b640 1 -- 172.21.15.133:0/2139847585 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f808c009d90 con 0x7f809809f910 2024-04-02T11:26:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8095d9b640 1 -- 172.21.15.133:0/2139847585 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f808c00e230 con 0x7f809809f910 2024-04-02T11:26:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.025+0000 7f8095d9b640 1 -- 172.21.15.133:0/2139847585 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f808c00e4d0 con 0x7f809809f910 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 -- 172.21.15.133:0/2139847585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809f910 msgr2=0x7f8098139b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 --2- 172.21.15.133:0/2139847585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809f910 0x7f8098139b70 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 -- 172.21.15.133:0/2139847585 shutdown_connections 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 --2- 172.21.15.133:0/2139847585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809f910 0x7f8098139b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 -- 172.21.15.133:0/2139847585 >> 172.21.15.133:0/2139847585 conn(0x7f80980a3830 msgr2=0x7f80980a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 -- 172.21.15.133:0/2139847585 shutdown_connections 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 1 -- 172.21.15.133:0/2139847585 wait complete. 2024-04-02T11:26:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.026+0000 7f809d0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.028+0000 7f809d0ed640 1 Processor -- start 2024-04-02T11:26:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.028+0000 7f809d0ed640 1 -- start start 2024-04-02T11:26:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f809d0ed640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f809d0ed640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8098124c20 con 0x7f809809edf0 2024-04-02T11:26:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f8096d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f8096d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46822/0 (socket says 172.21.15.133:46822) 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f8096d9d640 1 -- 172.21.15.133:0/3437112448 learned_addr learned my addr 172.21.15.133:0/3437112448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.029+0000 7f8096d9d640 1 -- 172.21.15.133:0/3437112448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098131f20 con 0x7f809809edf0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f8096d9d640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8088004cf0 con 0x7f809809edf0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8088004e50 con 0x7f809809edf0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8088007250 con 0x7f809809edf0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8098124e30 con 0x7f809809edf0 2024-04-02T11:26:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.030+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981243c0 con 0x7f809809edf0 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.031+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 45778+0+0 (crc 0 0 0) 0x7f8088007400 con 0x7f809809edf0 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.031+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098068e30 con 0x7f809809edf0 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.031+0000 7f807f7fe640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 0x7f808403ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.031+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2319+0+0 (crc 0 0 0) 0x7f8088048760 con 0x7f809809edf0 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.031+0000 7f809659c640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 0x7f808403ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.032+0000 7f809659c640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 0x7f808403ae00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:54.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.036+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f80880060b0 con 0x7f809809edf0 2024-04-02T11:26:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:54.215+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f8098063400 con 0x7f809809edf0 2024-04-02T11:26:55.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.103+0000 7f807f7fe640 1 -- 172.21.15.133:0/3437112448 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) ==== 109+0+0 (crc 0 0 0) 0x7f808800ad00 con 0x7f809809edf0 2024-04-02T11:26:55.103 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'rbd' created 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 msgr2=0x7f808403ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 0x7f808403ae00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 msgr2=0x7f8098120dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 shutdown_connections 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f80840388f0 0x7f808403ae00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 --2- 172.21.15.133:0/3437112448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f809809edf0 0x7f8098120dd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 >> 172.21.15.133:0/3437112448 conn(0x7f80980a3830 msgr2=0x7f80981342a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 shutdown_connections 2024-04-02T11:26:55.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.106+0000 7f809d0ed640 1 -- 172.21.15.133:0/3437112448 wait complete. 2024-04-02T11:26:55.120 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-02T11:26:55.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.233+0000 7f6aff91d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:55.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.243+0000 7f6aff91d640 1 Processor -- start 2024-04-02T11:26:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.244+0000 7f6aff91d640 1 -- start start 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aff91d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6af8137840 0x7f6af8139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aff91d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af80a0490 con 0x7f6af809f240 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aff91d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f6af809edf0 con 0x7f6af8137840 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afceb8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6af8137840 0x7f6af8139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afd6b9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6af809f240 0x7f6af809f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50828/0 (socket says 172.21.15.133:50828) 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afd6b9640 1 -- 172.21.15.133:0/689415159 learned_addr learned my addr 172.21.15.133:0/689415159 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afceb8640 1 -- 172.21.15.133:0/689415159 >> v1:172.21.15.133:6789/0 conn(0x7f6af809f240 legacy=0x7f6af809f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afceb8640 1 -- 172.21.15.133:0/689415159 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af80a1130 con 0x7f6af8137840 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aeffff640 1 -- 172.21.15.133:0/689415159 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2922433199 0 0) 0x7f6af80a0490 con 0x7f6af809f240 2024-04-02T11:26:55.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6afceb8640 1 --2- 172.21.15.133:0/689415159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af8137840 0x7f6af8139c60 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3962a1060ee95be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aeffff640 1 -- 172.21.15.133:0/689415159 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6af4009d90 con 0x7f6af8137840 2024-04-02T11:26:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aeffff640 1 -- 172.21.15.133:0/689415159 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6af400e230 con 0x7f6af8137840 2024-04-02T11:26:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.245+0000 7f6aeffff640 1 -- 172.21.15.133:0/689415159 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6af400e4d0 con 0x7f6af8137840 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 -- 172.21.15.133:0/689415159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af8137840 msgr2=0x7f6af8139c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 --2- 172.21.15.133:0/689415159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af8137840 0x7f6af8139c60 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 -- 172.21.15.133:0/689415159 shutdown_connections 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 --2- 172.21.15.133:0/689415159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af8137840 0x7f6af8139c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 -- 172.21.15.133:0/689415159 >> 172.21.15.133:0/689415159 conn(0x7f6af80a4a90 msgr2=0x7f6af80a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 -- 172.21.15.133:0/689415159 shutdown_connections 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.246+0000 7f6aff91d640 1 -- 172.21.15.133:0/689415159 wait complete. 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.247+0000 7f6aff91d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.248+0000 7f6aff91d640 1 Processor -- start 2024-04-02T11:26:55.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.248+0000 7f6aff91d640 1 -- start start 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aff91d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aff91d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8135280 con 0x7f6af809f240 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6afd6b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6afd6b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34990/0 (socket says 172.21.15.133:34990) 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6afd6b9640 1 -- 172.21.15.133:0/3789293549 learned_addr learned my addr 172.21.15.133:0/3789293549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6afd6b9640 1 -- 172.21.15.133:0/3789293549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af8121980 con 0x7f6af809f240 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6afd6b9640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6ae8004b80 con 0x7f6af809f240 2024-04-02T11:26:55.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6ae8004ce0 con 0x7f6af809f240 2024-04-02T11:26:55.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af8121f50 con 0x7f6af809f240 2024-04-02T11:26:55.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.249+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6ae8007120 con 0x7f6af809f240 2024-04-02T11:26:55.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.250+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af8121bb0 con 0x7f6af809f240 2024-04-02T11:26:55.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.250+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f6ae80072d0 con 0x7f6af809f240 2024-04-02T11:26:55.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.250+0000 7f6aedffb640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 0x7f6adc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:55.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.251+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2661+0+0 (crc 0 0 0) 0x7f6ae8048280 con 0x7f6af809f240 2024-04-02T11:26:55.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.251+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af8125ab0 con 0x7f6af809f240 2024-04-02T11:26:55.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.251+0000 7f6afceb8640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 0x7f6adc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:55.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.251+0000 7f6afceb8640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 0x7f6adc03ae90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:55.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.256+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f6ae800f230 con 0x7f6af809f240 2024-04-02T11:26:55.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:55.434+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7f6af8063400 con 0x7f6af809f240 2024-04-02T11:26:56.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.110+0000 7f6aedffb640 1 -- 172.21.15.133:0/3789293549 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) ==== 173+0+0 (crc 0 0 0) 0x7f6ae800a5d0 con 0x7f6af809f240 2024-04-02T11:26:56.111 INFO:teuthology.orchestra.run.smithi133.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 msgr2=0x7f6adc03ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 0x7f6adc03ae90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 msgr2=0x7f6af8122410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 shutdown_connections 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6adc038980 0x7f6adc03ae90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 --2- 172.21.15.133:0/3789293549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6af809f240 0x7f6af8122410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 >> 172.21.15.133:0/3789293549 conn(0x7f6af80a4a90 msgr2=0x7f6af8070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.113+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 shutdown_connections 2024-04-02T11:26:56.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.114+0000 7f6aff91d640 1 -- 172.21.15.133:0/3789293549 wait complete. 2024-04-02T11:26:56.133 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-02T11:26:56.134 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T11:26:56.134 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-02T11:26:56.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-02T11:26:56.135 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-02T11:26:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.262+0000 7fea9c39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.266+0000 7fea9c39d640 1 Processor -- start 2024-04-02T11:26:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.266+0000 7fea9c39d640 1 -- start start 2024-04-02T11:26:56.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.267+0000 7fea9c39d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fea9406e260 0x7fea94130cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.267+0000 7fea9c39d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea94130a60 con 0x7fea94131200 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.267+0000 7fea9c39d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fea9409c8b0 con 0x7fea9406e260 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.267+0000 7fea9a139640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fea9406e260 0x7fea94130cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99938640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fea94131200 0x7fea9409b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50842/0 (socket says 172.21.15.133:50842) 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99938640 1 -- 172.21.15.133:0/635149769 learned_addr learned my addr 172.21.15.133:0/635149769 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea9a139640 1 -- 172.21.15.133:0/635149769 >> v1:172.21.15.133:6789/0 conn(0x7fea94131200 legacy=0x7fea9409b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea9a139640 1 -- 172.21.15.133:0/635149769 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9409c2f0 con 0x7fea9406e260 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99137640 1 -- 172.21.15.133:0/635149769 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799229987 0 0) 0x7fea94130a60 con 0x7fea94131200 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea9a139640 1 --2- 172.21.15.133:0/635149769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea94130cc0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3649f1ad894c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99137640 1 -- 172.21.15.133:0/635149769 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fea90009d90 con 0x7fea9406e260 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99137640 1 -- 172.21.15.133:0/635149769 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fea9000e230 con 0x7fea9406e260 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.268+0000 7fea99137640 1 -- 172.21.15.133:0/635149769 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fea9000e4d0 con 0x7fea9406e260 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 -- 172.21.15.133:0/635149769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 msgr2=0x7fea94130cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 --2- 172.21.15.133:0/635149769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea94130cc0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 -- 172.21.15.133:0/635149769 shutdown_connections 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 --2- 172.21.15.133:0/635149769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea94130cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 -- 172.21.15.133:0/635149769 >> 172.21.15.133:0/635149769 conn(0x7fea940a0790 msgr2=0x7fea940a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 -- 172.21.15.133:0/635149769 shutdown_connections 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 1 -- 172.21.15.133:0/635149769 wait complete. 2024-04-02T11:26:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.269+0000 7fea9c39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.270+0000 7fea9c39d640 1 Processor -- start 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.270+0000 7fea9c39d640 1 -- start start 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.271+0000 7fea9c39d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.271+0000 7fea9c39d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea94120c90 con 0x7fea9406e260 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9a139640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9a139640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35006/0 (socket says 172.21.15.133:35006) 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9a139640 1 -- 172.21.15.133:0/577396971 learned_addr learned my addr 172.21.15.133:0/577396971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9a139640 1 -- 172.21.15.133:0/577396971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea941c90a0 con 0x7fea9406e260 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9a139640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fea90013610 con 0x7fea9406e260 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fea90013770 con 0x7fea9406e260 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea94072e00 con 0x7fea9406e260 2024-04-02T11:26:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fea90013a10 con 0x7fea9406e260 2024-04-02T11:26:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.272+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9411f0f0 con 0x7fea9406e260 2024-04-02T11:26:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.273+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fea90013bc0 con 0x7fea9406e260 2024-04-02T11:26:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.273+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea94120de0 con 0x7fea9406e260 2024-04-02T11:26:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.273+0000 7fea8affd640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 0x7fea7803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.273+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7fea90018040 con 0x7fea9406e260 2024-04-02T11:26:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.274+0000 7fea99938640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 0x7fea7803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.274+0000 7fea99938640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 0x7fea7803ae90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.279+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fea90018c30 con 0x7fea9406e260 2024-04-02T11:26:56.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.500+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fea94063400 con 0x7fea9406e260 2024-04-02T11:26:56.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.502+0000 7fea8affd640 1 -- 172.21.15.133:0/577396971 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87806 (crc 0 0 0) 0x7fea9000f2a0 con 0x7fea9406e260 2024-04-02T11:26:56.504 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 msgr2=0x7fea7803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 0x7fea7803ae90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 msgr2=0x7fea941c9460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 shutdown_connections 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fea78038980 0x7fea7803ae90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 --2- 172.21.15.133:0/577396971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea9406e260 0x7fea941c9460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 >> 172.21.15.133:0/577396971 conn(0x7fea940a0790 msgr2=0x7fea9412b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 shutdown_connections 2024-04-02T11:26:56.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.504+0000 7fea9c39d640 1 -- 172.21.15.133:0/577396971 wait complete. 2024-04-02T11:26:56.515 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1817191150},{"type":"v1","addr":"172.21.15.138:6801","nonce":1817191150}]},"active_addr":"172.21.15.138:6801/1817191150","active_change":"2024-04-02T11:26:48.564377+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":3970320120}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2572805094}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1947835436}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1566938795}]}]} 2024-04-02T11:26:56.517 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-02T11:26:56.517 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-02T11:26:56.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T11:26:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.623+0000 7f4d7e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.628+0000 7f4d7e31d640 1 Processor -- start 2024-04-02T11:26:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.628+0000 7f4d7e31d640 1 -- start start 2024-04-02T11:26:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d7e31d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d78099b30 0x7f4d781378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d7e31d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d78073a20 con 0x7f4d780725a0 2024-04-02T11:26:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d7e31d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f4d78099f40 con 0x7f4d78099b30 2024-04-02T11:26:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d78099b30 0x7f4d781378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d77fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4d780725a0 0x7f4d78099420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50846/0 (socket says 172.21.15.133:50846) 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.629+0000 7f4d77fff640 1 -- 172.21.15.133:0/1446682517 learned_addr learned my addr 172.21.15.133:0/1446682517 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d777fe640 1 -- 172.21.15.133:0/1446682517 >> v1:172.21.15.133:6789/0 conn(0x7f4d780725a0 legacy=0x7f4d78099420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d777fe640 1 -- 172.21.15.133:0/1446682517 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78073710 con 0x7f4d78099b30 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d76ffd640 1 -- 172.21.15.133:0/1446682517 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746292520 0 0) 0x7f4d78073a20 con 0x7f4d780725a0 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d777fe640 1 --2- 172.21.15.133:0/1446682517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d78099b30 0x7f4d781378b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d929e2e335396f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d76ffd640 1 -- 172.21.15.133:0/1446682517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d6c009d90 con 0x7f4d78099b30 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d76ffd640 1 -- 172.21.15.133:0/1446682517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d6c00e230 con 0x7f4d78099b30 2024-04-02T11:26:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d76ffd640 1 -- 172.21.15.133:0/1446682517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d6c00e4d0 con 0x7f4d78099b30 2024-04-02T11:26:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d7e31d640 1 -- 172.21.15.133:0/1446682517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d78099b30 msgr2=0x7f4d781378b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.630+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/1446682517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d78099b30 0x7f4d781378b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 1 -- 172.21.15.133:0/1446682517 shutdown_connections 2024-04-02T11:26:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/1446682517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d78099b30 0x7f4d781378b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 1 -- 172.21.15.133:0/1446682517 >> 172.21.15.133:0/1446682517 conn(0x7f4d780a0f90 msgr2=0x7f4d780a13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 1 -- 172.21.15.133:0/1446682517 shutdown_connections 2024-04-02T11:26:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 1 -- 172.21.15.133:0/1446682517 wait complete. 2024-04-02T11:26:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.631+0000 7f4d7e31d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.632+0000 7f4d7e31d640 1 Processor -- start 2024-04-02T11:26:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.632+0000 7f4d7e31d640 1 -- start start 2024-04-02T11:26:56.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.633+0000 7f4d7e31d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.633+0000 7f4d7e31d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78073db0 con 0x7f4d780725a0 2024-04-02T11:26:56.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.633+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.633+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35020/0 (socket says 172.21.15.133:35020) 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.633+0000 7f4d77fff640 1 -- 172.21.15.133:0/3995384716 learned_addr learned my addr 172.21.15.133:0/3995384716 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d77fff640 1 -- 172.21.15.133:0/3995384716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7812aaf0 con 0x7f4d780725a0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d77fff640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d68004b80 con 0x7f4d780725a0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d68004ce0 con 0x7f4d780725a0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7812bc00 con 0x7f4d780725a0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d68007120 con 0x7f4d780725a0 2024-04-02T11:26:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.634+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7812ff20 con 0x7f4d780725a0 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.635+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f4d680072d0 con 0x7f4d780725a0 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.635+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7812b640 con 0x7f4d780725a0 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.635+0000 7f4d74ff9640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 0x7f4d6403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.635+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f4d680482f0 con 0x7f4d780725a0 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.635+0000 7f4d777fe640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 0x7f4d6403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.636+0000 7f4d777fe640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 0x7f4d6403ae90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.640+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f4d6800a9e0 con 0x7f4d780725a0 2024-04-02T11:26:56.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.817+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d78063400 con 0x7f4d780725a0 2024-04-02T11:26:56.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.818+0000 7f4d74ff9640 1 -- 172.21.15.133:0/3995384716 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8755 (crc 0 0 0) 0x7f4d6800a5d0 con 0x7f4d780725a0 2024-04-02T11:26:56.818 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:56.819 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":14,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:26:56.103875+0000","last_up_change":"2024-04-02T11:26:50.078833+0000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T11:26:50.584783+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-02T11:26:54.219118+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6801","nonce":1793372331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1793372331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1793372331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":1793372331}]},"public_addr":"172.21.15.133:6801/1793372331","cluster_addr":"172.21.15.133:6802/1793372331","heartbeat_back_addr":"172.21.15.133:6806/1793372331","heartbeat_front_addr":"172.21.15.133:6807/1793372331","state":["exists","up"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":4121831248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4121831248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":4121831248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6813","nonce":4121831248}]},"public_addr":"172.21.15.133:6803/4121831248","cluster_addr":"172.21.15.133:6800/4121831248","heartbeat_back_addr":"172.21.15.133:6814/4121831248","heartbeat_front_addr":"172.21.15.133:6813/4121831248","state":["exists","up"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":1859851880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1859851880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":1859851880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1859851880}]},"public_addr":"172.21.15.138:6805/1859851880","cluster_addr":"172.21.15.138:6804/1859851880","heartbeat_back_addr":"172.21.15.138:6811/1859851880","heartbeat_front_addr":"172.21.15.138:6812/1859851880","state":["exists","up"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3056026735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3056026735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3056026735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":3056026735}]},"public_addr":"172.21.15.138:6806/3056026735","cluster_addr":"172.21.15.138:6807/3056026735","heartbeat_back_addr":"172.21.15.138:6808/3056026735","heartbeat_front_addr":"172.21.15.138:6809/3056026735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 msgr2=0x7f4d6403ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 0x7f4d6403ae90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 msgr2=0x7f4d78128150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 shutdown_connections 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f4d64038980 0x7f4d6403ae90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 --2- 172.21.15.133:0/3995384716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d780725a0 0x7f4d78128150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 >> 172.21.15.133:0/3995384716 conn(0x7f4d780a0f90 msgr2=0x7f4d7809c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 shutdown_connections 2024-04-02T11:26:56.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.820+0000 7f4d7e31d640 1 -- 172.21.15.133:0/3995384716 wait complete. 2024-04-02T11:26:56.831 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-02T11:26:56.831 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T11:26:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.938+0000 7f038b99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.942+0000 7f038b99d640 1 Processor -- start 2024-04-02T11:26:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.942+0000 7f038b99d640 1 -- start start 2024-04-02T11:26:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.943+0000 7f038b99d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f038406e260 0x7f0384130cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.943+0000 7f038b99d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0384130a60 con 0x7f0384131200 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.943+0000 7f038b99d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f038409c8b0 con 0x7f038406e260 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0389739640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f038406e260 0x7f0384130cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0388f38640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0384131200 0x7f038409b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50852/0 (socket says 172.21.15.133:50852) 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0388f38640 1 -- 172.21.15.133:0/2081087774 learned_addr learned my addr 172.21.15.133:0/2081087774 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0389739640 1 -- 172.21.15.133:0/2081087774 >> v1:172.21.15.133:6789/0 conn(0x7f0384131200 legacy=0x7f038409b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0389739640 1 -- 172.21.15.133:0/2081087774 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038409c2f0 con 0x7f038406e260 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f0389739640 1 --2- 172.21.15.133:0/2081087774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f0384130cc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a710f6adf948f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f037bfff640 1 -- 172.21.15.133:0/2081087774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f038000e040 con 0x7f038406e260 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f037bfff640 1 -- 172.21.15.133:0/2081087774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0380007670 con 0x7f038406e260 2024-04-02T11:26:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.944+0000 7f037bfff640 1 -- 172.21.15.133:0/2081087774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0380007910 con 0x7f038406e260 2024-04-02T11:26:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 -- 172.21.15.133:0/2081087774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 msgr2=0x7f0384130cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 --2- 172.21.15.133:0/2081087774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f0384130cc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 -- 172.21.15.133:0/2081087774 shutdown_connections 2024-04-02T11:26:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 --2- 172.21.15.133:0/2081087774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f0384130cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 -- 172.21.15.133:0/2081087774 >> 172.21.15.133:0/2081087774 conn(0x7f03840a0790 msgr2=0x7f03840a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 -- 172.21.15.133:0/2081087774 shutdown_connections 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 1 -- 172.21.15.133:0/2081087774 wait complete. 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.945+0000 7f038b99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.946+0000 7f038b99d640 1 Processor -- start 2024-04-02T11:26:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.947+0000 7f038b99d640 1 -- start start 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f038b99d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f038b99d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0384135790 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0389739640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0389739640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35040/0 (socket says 172.21.15.133:35040) 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0389739640 1 -- 172.21.15.133:0/3211251256 learned_addr learned my addr 172.21.15.133:0/3211251256 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0389739640 1 -- 172.21.15.133:0/3211251256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0384136540 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0389739640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f038000e040 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038406aa20 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0380012a80 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0380012d20 con 0x7f038406e260 2024-04-02T11:26:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.948+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0384136110 con 0x7f038406e260 2024-04-02T11:26:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.949+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f03800153f0 con 0x7f038406e260 2024-04-02T11:26:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.949+0000 7f0379ffb640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 0x7f036803ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.949+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0384063400 con 0x7f038406e260 2024-04-02T11:26:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.949+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2672+0+0 (crc 0 0 0) 0x7f0380049aa0 con 0x7f038406e260 2024-04-02T11:26:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.950+0000 7f0388f38640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 0x7f036803ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.950+0000 7f0388f38640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 0x7f036803ae40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:56.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:56.955+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f0380049e60 con 0x7f038406e260 2024-04-02T11:26:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.131+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0384134930 con 0x7f038406e260 2024-04-02T11:26:57.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.132+0000 7f0379ffb640 1 -- 172.21.15.133:0/3211251256 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+8746 (crc 0 0 0) 0x7f038000fe40 con 0x7f038406e260 2024-04-02T11:26:57.132 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:57.133 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":15,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:26:57.111799+0000","last_up_change":"2024-04-02T11:26:50.078833+0000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T11:26:50.584783+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-02T11:26:54.219118+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6801","nonce":1793372331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1793372331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1793372331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":1793372331}]},"public_addr":"172.21.15.133:6801/1793372331","cluster_addr":"172.21.15.133:6802/1793372331","heartbeat_back_addr":"172.21.15.133:6806/1793372331","heartbeat_front_addr":"172.21.15.133:6807/1793372331","state":["exists","up"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":4121831248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4121831248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":4121831248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6813","nonce":4121831248}]},"public_addr":"172.21.15.133:6803/4121831248","cluster_addr":"172.21.15.133:6800/4121831248","heartbeat_back_addr":"172.21.15.133:6814/4121831248","heartbeat_front_addr":"172.21.15.133:6813/4121831248","state":["exists","up"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":1859851880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1859851880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":1859851880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1859851880}]},"public_addr":"172.21.15.138:6805/1859851880","cluster_addr":"172.21.15.138:6804/1859851880","heartbeat_back_addr":"172.21.15.138:6811/1859851880","heartbeat_front_addr":"172.21.15.138:6812/1859851880","state":["exists","up"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3056026735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3056026735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3056026735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":3056026735}]},"public_addr":"172.21.15.138:6806/3056026735","cluster_addr":"172.21.15.138:6807/3056026735","heartbeat_back_addr":"172.21.15.138:6808/3056026735","heartbeat_front_addr":"172.21.15.138:6809/3056026735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T11:26:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.134+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 msgr2=0x7f036803ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.134+0000 7f038b99d640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 0x7f036803ae40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.134+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 msgr2=0x7f038412a340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.134+0000 7f038b99d640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 shutdown_connections 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f0368038930 0x7f036803ae40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 --2- 172.21.15.133:0/3211251256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f038406e260 0x7f038412a340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 >> 172.21.15.133:0/3211251256 conn(0x7f03840a0790 msgr2=0x7f038409a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 shutdown_connections 2024-04-02T11:26:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.135+0000 7f038b99d640 1 -- 172.21.15.133:0/3211251256 wait complete. 2024-04-02T11:26:57.146 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-02T11:26:57.147 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-02T11:26:57.147 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-02T11:26:57.147 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-02T11:26:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.265+0000 7f6774e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.265+0000 7f939b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.265+0000 7f95e915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.265+0000 7f2d9731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.270+0000 7f6774e6d640 1 Processor -- start 2024-04-02T11:26:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.270+0000 7f95e915d640 1 Processor -- start 2024-04-02T11:26:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.270+0000 7f939b26d640 1 Processor -- start 2024-04-02T11:26:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.271+0000 7f6774e6d640 1 -- start start 2024-04-02T11:26:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.271+0000 7f95e915d640 1 -- start start 2024-04-02T11:26:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.271+0000 7f2d9731d640 1 Processor -- start 2024-04-02T11:26:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.271+0000 7f939b26d640 1 -- start start 2024-04-02T11:26:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.271+0000 7f2d9731d640 1 -- start start 2024-04-02T11:26:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f6774e6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6770131780 0x7f6770131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f939b26d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939409c820 0x7f9394131840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f6774e6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67700a6af0 con 0x7f677009edf0 2024-04-02T11:26:57.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f6774e6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f6770073660 con 0x7f6770131780 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f939b26d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939406d300 con 0x7f9394131d80 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f939b26d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f939409cce0 con 0x7f939409c820 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f676e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6770131780 0x7f6770131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f676dd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f677009edf0 0x7f677009f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50866/0 (socket says 172.21.15.133:50866) 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f676dd9b640 1 -- 172.21.15.133:0/1471840178 learned_addr learned my addr 172.21.15.133:0/1471840178 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f9399009640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939409c820 0x7f9394131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f9399009640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939409c820 0x7f9394131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35042/0 (socket says 172.21.15.133:35042) 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f9399009640 1 -- 172.21.15.133:0/4054077360 learned_addr learned my addr 172.21.15.133:0/4054077360 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f676e59c640 1 -- 172.21.15.133:0/1471840178 >> v1:172.21.15.133:6789/0 conn(0x7f677009edf0 legacy=0x7f677009f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f676e59c640 1 -- 172.21.15.133:0/1471840178 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700a67e0 con 0x7f6770131780 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f95e915d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95e41313d0 0x7f95e4135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f95e915d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e4130a60 con 0x7f95e406e260 2024-04-02T11:26:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.272+0000 7f95e915d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f95e409c8b0 con 0x7f95e41313d0 2024-04-02T11:26:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f9399009640 1 -- 172.21.15.133:0/4054077360 >> v1:172.21.15.133:6789/0 conn(0x7f9394131d80 legacy=0x7f939406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f9399009640 1 -- 172.21.15.133:0/4054077360 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939409c380 con 0x7f939409c820 2024-04-02T11:26:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95e41313d0 0x7f95e4135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e2d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f95e406e260 0x7f95e4130cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50876/0 (socket says 172.21.15.133:50876) 2024-04-02T11:26:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e2d9d640 1 -- 172.21.15.133:0/621176676 learned_addr learned my addr 172.21.15.133:0/621176676 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f676e59c640 1 --2- 172.21.15.133:0/1471840178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6770131780 0x7f6770131b90 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ad989ba548e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d9731d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d900a4530 0x7f2d9012f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d9731d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d90072410 con 0x7f2d9012fb40 2024-04-02T11:26:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f9399009640 1 --2- 172.21.15.133:0/4054077360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939409c820 0x7f9394131840 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33e59bf77f65907 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d9731d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9009ed30 con 0x7f2d900a4530 2024-04-02T11:26:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d950b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d900a4530 0x7f2d9012f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d950b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d900a4530 0x7f2d9012f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35078/0 (socket says 172.21.15.133:35078) 2024-04-02T11:26:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d950b9640 1 -- 172.21.15.133:0/2777180890 learned_addr learned my addr 172.21.15.133:0/2777180890 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e259c640 1 -- 172.21.15.133:0/621176676 >> v1:172.21.15.133:6789/0 conn(0x7f95e406e260 legacy=0x7f95e4130cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e259c640 1 -- 172.21.15.133:0/621176676 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e409c2f0 con 0x7f95e41313d0 2024-04-02T11:26:57.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f676d59a640 1 -- 172.21.15.133:0/1471840178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6760009d90 con 0x7f6770131780 2024-04-02T11:26:57.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f938bfff640 1 -- 172.21.15.133:0/4054077360 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9390009e50 con 0x7f939409c820 2024-04-02T11:26:57.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f95e259c640 1 --2- 172.21.15.133:0/621176676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e41313d0 0x7f95e4135dc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5451b83ab0e6cf25 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f676d59a640 1 -- 172.21.15.133:0/1471840178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f676000e230 con 0x7f6770131780 2024-04-02T11:26:57.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f938bfff640 1 -- 172.21.15.133:0/4054077360 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9390010440 con 0x7f939409c820 2024-04-02T11:26:57.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f938bfff640 1 -- 172.21.15.133:0/4054077360 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f93900106e0 con 0x7f939409c820 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d950b9640 1 -- 172.21.15.133:0/2777180890 >> v1:172.21.15.133:6789/0 conn(0x7f2d9012fb40 legacy=0x7f2d900706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.273+0000 7f2d950b9640 1 -- 172.21.15.133:0/2777180890 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d90072100 con 0x7f2d900a4530 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f2d950b9640 1 --2- 172.21.15.133:0/2777180890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d900a4530 0x7f2d9012f600 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b958c52d809212f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f2d87fff640 1 -- 172.21.15.133:0/2777180890 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d8c009e50 con 0x7f2d900a4530 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f95e1d9b640 1 -- 172.21.15.133:0/621176676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95d8009d90 con 0x7f95e41313d0 2024-04-02T11:26:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f95e1d9b640 1 -- 172.21.15.133:0/621176676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f95d800e230 con 0x7f95e41313d0 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f95e1d9b640 1 -- 172.21.15.133:0/621176676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95d800e4d0 con 0x7f95e41313d0 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f2d87fff640 1 -- 172.21.15.133:0/2777180890 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2d8c00e230 con 0x7f2d900a4530 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f2d87fff640 1 -- 172.21.15.133:0/2777180890 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d8c00e4d0 con 0x7f2d900a4530 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f6774e6d640 1 -- 172.21.15.133:0/1471840178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6770131780 msgr2=0x7f6770131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f6774e6d640 1 --2- 172.21.15.133:0/1471840178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6770131780 0x7f6770131b90 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f6774e6d640 1 -- 172.21.15.133:0/1471840178 shutdown_connections 2024-04-02T11:26:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f6774e6d640 1 --2- 172.21.15.133:0/1471840178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6770131780 0x7f6770131b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f6774e6d640 1 -- 172.21.15.133:0/1471840178 >> 172.21.15.133:0/1471840178 conn(0x7f67700a4ba0 msgr2=0x7f67700a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.274+0000 7f939b26d640 1 -- 172.21.15.133:0/4054077360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939409c820 msgr2=0x7f9394131840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 --2- 172.21.15.133:0/4054077360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939409c820 0x7f9394131840 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f6774e6d640 1 -- 172.21.15.133:0/1471840178 shutdown_connections 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f6774e6d640 1 -- 172.21.15.133:0/1471840178 wait complete. 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 -- 172.21.15.133:0/4054077360 shutdown_connections 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 --2- 172.21.15.133:0/4054077360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939409c820 0x7f9394131840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 -- 172.21.15.133:0/4054077360 >> 172.21.15.133:0/4054077360 conn(0x7f93940a0790 msgr2=0x7f93940a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 -- 172.21.15.133:0/2777180890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d900a4530 msgr2=0x7f2d9012f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2777180890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d900a4530 0x7f2d9012f600 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 -- 172.21.15.133:0/621176676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e41313d0 msgr2=0x7f95e4135dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 --2- 172.21.15.133:0/621176676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e41313d0 0x7f95e4135dc0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 -- 172.21.15.133:0/4054077360 shutdown_connections 2024-04-02T11:26:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f6774e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 1 -- 172.21.15.133:0/4054077360 wait complete. 2024-04-02T11:26:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 -- 172.21.15.133:0/2777180890 shutdown_connections 2024-04-02T11:26:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 -- 172.21.15.133:0/621176676 shutdown_connections 2024-04-02T11:26:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 --2- 172.21.15.133:0/621176676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e41313d0 0x7f95e4135dc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 -- 172.21.15.133:0/621176676 >> 172.21.15.133:0/621176676 conn(0x7f95e40a0790 msgr2=0x7f95e40a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2777180890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d900a4530 0x7f2d9012f600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 -- 172.21.15.133:0/2777180890 >> 172.21.15.133:0/2777180890 conn(0x7f2d900a2920 msgr2=0x7f2d900a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 -- 172.21.15.133:0/2777180890 shutdown_connections 2024-04-02T11:26:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 1 -- 172.21.15.133:0/2777180890 wait complete. 2024-04-02T11:26:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f939b26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 -- 172.21.15.133:0/621176676 shutdown_connections 2024-04-02T11:26:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 1 -- 172.21.15.133:0/621176676 wait complete. 2024-04-02T11:26:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f95e915d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.275+0000 7f2d9731d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.276+0000 7f6774e6d640 1 Processor -- start 2024-04-02T11:26:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.276+0000 7f939b26d640 1 Processor -- start 2024-04-02T11:26:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.276+0000 7f6774e6d640 1 -- start start 2024-04-02T11:26:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.276+0000 7f939b26d640 1 -- start start 2024-04-02T11:26:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.277+0000 7f2d9731d640 1 Processor -- start 2024-04-02T11:26:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.277+0000 7f2d9731d640 1 -- start start 2024-04-02T11:26:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.277+0000 7f95e915d640 1 Processor -- start 2024-04-02T11:26:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.277+0000 7f95e915d640 1 -- start start 2024-04-02T11:26:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f6774e6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f939b26d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f939b26d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f939411cdd0 con 0x7f9394131d80 2024-04-02T11:26:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f6774e6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6770120910 con 0x7f677009edf0 2024-04-02T11:26:57.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f676e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d9731d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f676e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35082/0 (socket says 172.21.15.133:35082) 2024-04-02T11:26:57.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f676e59c640 1 -- 172.21.15.133:0/3448056684 learned_addr learned my addr 172.21.15.133:0/3448056684 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f9399009640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f9399009640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35096/0 (socket says 172.21.15.133:35096) 2024-04-02T11:26:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f9399009640 1 -- 172.21.15.133:0/3258060091 learned_addr learned my addr 172.21.15.133:0/3258060091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d9731d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9011ec90 con 0x7f2d9012fb40 2024-04-02T11:26:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d950b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d950b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35102/0 (socket says 172.21.15.133:35102) 2024-04-02T11:26:57.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d950b9640 1 -- 172.21.15.133:0/2442755262 learned_addr learned my addr 172.21.15.133:0/2442755262 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f676e59c640 1 -- 172.21.15.133:0/3448056684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6770120b90 con 0x7f677009edf0 2024-04-02T11:26:57.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f9399009640 1 -- 172.21.15.133:0/3258060091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939411e3b0 con 0x7f9394131d80 2024-04-02T11:26:57.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f676e59c640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.278+0000 7f2d950b9640 1 -- 172.21.15.133:0/2442755262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d900693b0 con 0x7f2d9012fb40 2024-04-02T11:26:57.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f9399009640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f95e915d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6760015760 con 0x7f677009edf0 2024-04-02T11:26:57.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677006aa20 con 0x7f677009edf0 2024-04-02T11:26:57.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f95e915d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4119d20 con 0x7f95e406e260 2024-04-02T11:26:57.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f2d950b9640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f67600158c0 con 0x7f677009edf0 2024-04-02T11:26:57.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6760015b60 con 0x7f677009edf0 2024-04-02T11:26:57.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d8c00ee50 con 0x7f2d9012fb40 2024-04-02T11:26:57.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6770120440 con 0x7f677009edf0 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9390017040 con 0x7f9394131d80 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f93900121e0 con 0x7f9394131d80 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f93900107c0 con 0x7f9394131d80 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939411ec90 con 0x7f9394131d80 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9012f030 con 0x7f2d9012fb40 2024-04-02T11:26:57.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9011c710 con 0x7f2d9012fb40 2024-04-02T11:26:57.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.279+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941203b0 con 0x7f9394131d80 2024-04-02T11:26:57.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f677006e4c0 con 0x7f677009edf0 2024-04-02T11:26:57.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f95e2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f95e2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35110/0 (socket says 172.21.15.133:35110) 2024-04-02T11:26:57.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f95e2d9d640 1 -- 172.21.15.133:0/3979215354 learned_addr learned my addr 172.21.15.133:0/3979215354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2d8c015420 con 0x7f2d9012fb40 2024-04-02T11:26:57.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.280+0000 7f95e2d9d640 1 -- 172.21.15.133:0/3979215354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e411bfa0 con 0x7f95e406e260 2024-04-02T11:26:57.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f9390010920 con 0x7f9394131d80 2024-04-02T11:26:57.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95e2d9d640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f676000e360 con 0x7f677009edf0 2024-04-02T11:26:57.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d8c0156c0 con 0x7f2d9012fb40 2024-04-02T11:26:57.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d70000f80 con 0x7f2d9012fb40 2024-04-02T11:26:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f9389ffb640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 0x7f937803f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f9390048d30 con 0x7f9394131d80 2024-04-02T11:26:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f2d8c015820 con 0x7f2d9012fb40 2024-04-02T11:26:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f675effd640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 0x7f675003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95d4005a60 con 0x7f95e406e260 2024-04-02T11:26:57.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 0x7f93941c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 --> v2:172.21.15.138:6805/1859851880 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f939411cc50 con 0x7f93941c4820 2024-04-02T11:26:57.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f95d4005bc0 con 0x7f95e406e260 2024-04-02T11:26:57.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95e915d640 1 -- 172.21.15.133:0/3979215354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e411baf0 con 0x7f95e406e260 2024-04-02T11:26:57.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f95d4005e60 con 0x7f95e406e260 2024-04-02T11:26:57.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f6760049c70 con 0x7f677009edf0 2024-04-02T11:26:57.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f676dd9b640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 0x7f675003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f95e915d640 1 -- 172.21.15.133:0/3979215354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e411d690 con 0x7f95e406e260 2024-04-02T11:26:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.281+0000 7f2d85ffb640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 0x7f2d7403aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f2d8c0496f0 con 0x7f2d9012fb40 2024-04-02T11:26:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f2d948b8640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 0x7f2d7403aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f9398808640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 0x7f937803f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f675effd640 1 --2- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 0x7f675003fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 --> v2:172.21.15.133:6803/4121831248 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67500404f0 con 0x7f675003d9d0 2024-04-02T11:26:57.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f67600123f0 con 0x7f677009edf0 2024-04-02T11:26:57.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f2d85ffb640 1 --2- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 0x7f2d7403fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 --> v2:172.21.15.133:6801/1793372331 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d740404f0 con 0x7f2d7403d9d0 2024-04-02T11:26:57.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f2d8c04bb10 con 0x7f2d9012fb40 2024-04-02T11:26:57.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f676dd9b640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 0x7f675003aee0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f95e915d640 1 -- 172.21.15.133:0/3979215354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f95e4119b20 con 0x7f95e406e260 2024-04-02T11:26:57.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f95d40070b0 con 0x7f95e406e260 2024-04-02T11:26:57.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.282+0000 7f939980a640 1 --2- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 0x7f93941c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T11:26:57.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95cb7fe640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 0x7f95d003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f2d948b8640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 0x7f2d7403aee0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f9398808640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 0x7f937803f340 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f95d40481a0 con 0x7f95e406e260 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95cb7fe640 1 --2- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 0x7f95d003fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95e259c640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 0x7f95d003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 --> v2:172.21.15.138:6806/3056026735 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95d00404f0 con 0x7f95d003d9d0 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f95d400da20 con 0x7f95e406e260 2024-04-02T11:26:57.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f676ed9d640 1 --2- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 0x7f675003fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T11:26:57.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.283+0000 7f2d958ba640 1 --2- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 0x7f2d7403fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T11:26:57.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.284+0000 7f95e259c640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 0x7f95d003aee0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.284+0000 7f95e359e640 1 --2- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 0x7f95d003fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T11:26:57.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.287+0000 7f95e359e640 1 --2- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 0x7f95d003fe60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.287+0000 7f939980a640 1 --2- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 0x7f93941c4c30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.288+0000 7f2d958ba640 1 --2- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 0x7f2d7403fe60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.288+0000 7f676ed9d640 1 --2- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 0x7f675003fe60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.288+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== osd.3 v2:172.21.15.138:6806/3056026735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f95d00404f0 con 0x7f95d003d9d0 2024-04-02T11:26:57.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.289+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== osd.2 v2:172.21.15.138:6805/1859851880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f938c00a820 con 0x7f93941c4820 2024-04-02T11:26:57.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.290+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== osd.0 v2:172.21.15.133:6801/1793372331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f2d740404f0 con 0x7f2d7403d9d0 2024-04-02T11:26:57.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.290+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== osd.1 v2:172.21.15.133:6803/4121831248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f67500404f0 con 0x7f675003d9d0 2024-04-02T11:26:57.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.292+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 --> v2:172.21.15.138:6805/1859851880 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9394063400 con 0x7f93941c4820 2024-04-02T11:26:57.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.293+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 --> v2:172.21.15.133:6801/1793372331 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d70002c30 con 0x7f2d7403d9d0 2024-04-02T11:26:57.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.294+0000 7f9389ffb640 1 -- 172.21.15.133:0/3258060091 <== osd.2 v2:172.21.15.138:6805/1859851880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9394063400 con 0x7f93941c4820 2024-04-02T11:26:57.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.294+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 msgr2=0x7f93941c4c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.294+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 0x7f93941c4c30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.294+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 msgr2=0x7f937803f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.294+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 0x7f937803f340 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 msgr2=0x7f939411d320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 shutdown_connections 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> v2:172.21.15.138:6805/1859851880 conn(0x7f93941c4820 0x7f93941c4c30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f937803ce30 0x7f937803f340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 --2- 172.21.15.133:0/3258060091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9394131d80 0x7f939411d320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 >> 172.21.15.133:0/3258060091 conn(0x7f93940a0790 msgr2=0x7f939406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f2d85ffb640 1 -- 172.21.15.133:0/2442755262 <== osd.0 v2:172.21.15.133:6801/1793372331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d70002c30 con 0x7f2d7403d9d0 2024-04-02T11:26:57.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 shutdown_connections 2024-04-02T11:26:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f939b26d640 1 -- 172.21.15.133:0/3258060091 wait complete. 2024-04-02T11:26:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f95e915d640 1 -- 172.21.15.133:0/3979215354 --> v2:172.21.15.138:6806/3056026735 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95e409a9e0 con 0x7f95d003d9d0 2024-04-02T11:26:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 msgr2=0x7f2d7403fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.295+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 0x7f2d7403fe60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 msgr2=0x7f2d7403aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 0x7f2d7403aee0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 msgr2=0x7f2d9011b070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 shutdown_connections 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> v2:172.21.15.133:6801/1793372331 conn(0x7f2d7403d9d0 0x7f2d7403fe60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2d740389d0 0x7f2d7403aee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 --2- 172.21.15.133:0/2442755262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9012fb40 0x7f2d9011b070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 >> 172.21.15.133:0/2442755262 conn(0x7f2d900a2920 msgr2=0x7f2d901ba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 shutdown_connections 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f2d9731d640 1 -- 172.21.15.133:0/2442755262 wait complete. 2024-04-02T11:26:57.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.296+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 --> v2:172.21.15.133:6803/4121831248 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6770063400 con 0x7f675003d9d0 2024-04-02T11:26:57.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.297+0000 7f95cb7fe640 1 -- 172.21.15.133:0/3979215354 <== osd.3 v2:172.21.15.138:6806/3056026735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f95dc00a820 con 0x7f95d003d9d0 2024-04-02T11:26:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.297+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 msgr2=0x7f95d003fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.297+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 0x7f95d003fe60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 msgr2=0x7f95d003aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 0x7f95d003aee0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 msgr2=0x7f95e411a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 shutdown_connections 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> v2:172.21.15.138:6806/3056026735 conn(0x7f95d003d9d0 0x7f95d003fe60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f95d00389d0 0x7f95d003aee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 --2- 172.21.15.133:0/3979215354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e406e260 0x7f95e411a040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 >> 172.21.15.133:0/3979215354 conn(0x7f95e40a0790 msgr2=0x7f95e412eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 shutdown_connections 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.298+0000 7f95c97fa640 1 -- 172.21.15.133:0/3979215354 wait complete. 2024-04-02T11:26:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.305+0000 7f675effd640 1 -- 172.21.15.133:0/3448056684 <== osd.1 v2:172.21.15.133:6803/4121831248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6770063400 con 0x7f675003d9d0 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 msgr2=0x7f675003fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 0x7f675003fe60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 msgr2=0x7f675003aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 0x7f675003aee0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 msgr2=0x7f677011d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 shutdown_connections 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f67500389d0 0x7f675003aee0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> v2:172.21.15.133:6803/4121831248 conn(0x7f675003d9d0 0x7f675003fe60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 --2- 172.21.15.133:0/3448056684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677009edf0 0x7f677011d2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.306+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 >> 172.21.15.133:0/3448056684 conn(0x7f67700a4ba0 msgr2=0x7f67701bc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.307+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 shutdown_connections 2024-04-02T11:26:57.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.307+0000 7f6774e6d640 1 -- 172.21.15.133:0/3448056684 wait complete. 2024-04-02T11:26:57.327 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.327 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-02T11:26:57.328 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.328 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T11:26:57.329 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-02T11:26:57.330 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-02T11:26:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.485+0000 7f225c8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.485+0000 7f79a3c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.485+0000 7f5d7c55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.485+0000 7fa529a24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f5d7c55d640 1 Processor -- start 2024-04-02T11:26:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f79a3c2d640 1 Processor -- start 2024-04-02T11:26:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f225c8ad640 1 Processor -- start 2024-04-02T11:26:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f225c8ad640 1 -- start start 2024-04-02T11:26:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f5d7c55d640 1 -- start start 2024-04-02T11:26:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7fa529a24640 1 Processor -- start 2024-04-02T11:26:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7fa529a24640 1 -- start start 2024-04-02T11:26:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.490+0000 7f79a3c2d640 1 -- start start 2024-04-02T11:26:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.491+0000 7fa529a24640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5240a4530 0x7fa52412f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.491+0000 7fa529a24640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa524072410 con 0x7fa52412fb40 2024-04-02T11:26:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.491+0000 7fa529a24640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fa52409ed30 con 0x7fa5240a4530 2024-04-02T11:26:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225c8ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2258137950 0x7f2258139d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa522ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5240a4530 0x7fa52412f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa5227fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa52412fb40 0x7fa5240706b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50894/0 (socket says 172.21.15.133:50894) 2024-04-02T11:26:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa5227fc640 1 -- 172.21.15.133:0/204263095 learned_addr learned my addr 172.21.15.133:0/204263095 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2258131780 0x7f2258131b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50906/0 (socket says 172.21.15.133:50906) 2024-04-02T11:26:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f2255d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2258137950 0x7f2258139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225c8ad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22581322a0 con 0x7f2258131780 2024-04-02T11:26:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225c8ad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f22580a6af0 con 0x7f2258137950 2024-04-02T11:26:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225659c640 1 -- 172.21.15.133:0/4104713597 learned_addr learned my addr 172.21.15.133:0/4104713597 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa522ffd640 1 -- 172.21.15.133:0/204263095 >> v1:172.21.15.133:6789/0 conn(0x7fa52412fb40 legacy=0x7fa5240706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa521ffb640 1 -- 172.21.15.133:0/204263095 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440679083 0 0) 0x7fa524072410 con 0x7fa52412fb40 2024-04-02T11:26:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225559a640 1 -- 172.21.15.133:0/4104713597 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 60155453 0 0) 0x7f2248001110 con 0x7f2258131780 2024-04-02T11:26:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7fa522ffd640 1 -- 172.21.15.133:0/204263095 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa524072100 con 0x7fa5240a4530 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f225559a640 1 -- 172.21.15.133:0/4104713597 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2244003580 con 0x7f2258131780 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f2255d9b640 1 -- 172.21.15.133:0/4104713597 >> v1:172.21.15.133:6789/0 conn(0x7f2258131780 legacy=0x7f2258131b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f2255d9b640 1 -- 172.21.15.133:0/4104713597 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2258073660 con 0x7f2258137950 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.492+0000 7f2255d9b640 1 --2- 172.21.15.133:0/4104713597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258137950 0x7f2258139d70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05d860f56a677ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7fa522ffd640 1 --2- 172.21.15.133:0/204263095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52412f600 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9e8f9a755554f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f225559a640 1 -- 172.21.15.133:0/4104713597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2240009d00 con 0x7f2258137950 2024-04-02T11:26:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a3c2d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f799c131780 0x7f799c131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f225559a640 1 -- 172.21.15.133:0/4104713597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2240009e60 con 0x7f2258137950 2024-04-02T11:26:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f225559a640 1 -- 172.21.15.133:0/4104713597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f224000e360 con 0x7f2258137950 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7fa521ffb640 1 -- 172.21.15.133:0/204263095 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa514009d90 con 0x7fa5240a4530 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7fa521ffb640 1 -- 172.21.15.133:0/204263095 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa51400e230 con 0x7fa5240a4530 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d7c55d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5d7406d890 0x7f5d74136130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d7c55d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7406d300 con 0x7f5d74130a00 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d7c55d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7409c170 con 0x7f5d7406d890 2024-04-02T11:26:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7fa521ffb640 1 -- 172.21.15.133:0/204263095 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa51400e4d0 con 0x7fa5240a4530 2024-04-02T11:26:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a19c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f799c131780 0x7f799c131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a3c2d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799c0a6af0 con 0x7f799c09edf0 2024-04-02T11:26:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a3c2d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f799c073660 con 0x7f799c131780 2024-04-02T11:26:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a19c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f799c131780 0x7f799c131b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35132/0 (socket says 172.21.15.133:35132) 2024-04-02T11:26:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d79af8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5d7406d890 0x7f5d74136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d7a2f9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5d74130a00 0x7f5d74130e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50928/0 (socket says 172.21.15.133:50928) 2024-04-02T11:26:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a19c9640 1 -- 172.21.15.133:0/803754179 learned_addr learned my addr 172.21.15.133:0/803754179 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f5d7a2f9640 1 -- 172.21.15.133:0/2729492610 learned_addr learned my addr 172.21.15.133:0/2729492610 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a19c9640 1 -- 172.21.15.133:0/803754179 >> v1:172.21.15.133:6789/0 conn(0x7f799c09edf0 legacy=0x7f799c09f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.493+0000 7f79a19c9640 1 -- 172.21.15.133:0/803754179 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c0a67e0 con 0x7f799c131780 2024-04-02T11:26:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d79af8640 1 -- 172.21.15.133:0/2729492610 >> v1:172.21.15.133:6789/0 conn(0x7f5d74130a00 legacy=0x7f5d74130e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d79af8640 1 -- 172.21.15.133:0/2729492610 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d74130520 con 0x7f5d7406d890 2024-04-02T11:26:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f79a19c9640 1 --2- 172.21.15.133:0/803754179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c131780 0x7f799c131b90 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4bfe03a6f591f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d79af8640 1 --2- 172.21.15.133:0/2729492610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d74136130 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec9a2d6949f8592 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f79a09c7640 1 -- 172.21.15.133:0/803754179 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7998009e50 con 0x7f799c131780 2024-04-02T11:26:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f79a09c7640 1 -- 172.21.15.133:0/803754179 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f799800e230 con 0x7f799c131780 2024-04-02T11:26:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d792f7640 1 -- 172.21.15.133:0/2729492610 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d64009d90 con 0x7f5d7406d890 2024-04-02T11:26:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f79a09c7640 1 -- 172.21.15.133:0/803754179 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f799800e4d0 con 0x7f799c131780 2024-04-02T11:26:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 -- 172.21.15.133:0/4104713597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258137950 msgr2=0x7f2258139d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 --2- 172.21.15.133:0/4104713597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258137950 0x7f2258139d70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d792f7640 1 -- 172.21.15.133:0/2729492610 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d6400e230 con 0x7f5d7406d890 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f5d792f7640 1 -- 172.21.15.133:0/2729492610 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d64009d90 con 0x7f5d7406d890 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 -- 172.21.15.133:0/4104713597 shutdown_connections 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 --2- 172.21.15.133:0/4104713597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258137950 0x7f2258139d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 -- 172.21.15.133:0/4104713597 >> 172.21.15.133:0/4104713597 conn(0x7f22580a4ba0 msgr2=0x7f22580a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 -- 172.21.15.133:0/4104713597 shutdown_connections 2024-04-02T11:26:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 1 -- 172.21.15.133:0/4104713597 wait complete. 2024-04-02T11:26:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 -- 172.21.15.133:0/204263095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 msgr2=0x7fa52412f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 --2- 172.21.15.133:0/204263095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52412f600 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 -- 172.21.15.133:0/204263095 shutdown_connections 2024-04-02T11:26:57.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 --2- 172.21.15.133:0/204263095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52412f600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 -- 172.21.15.133:0/204263095 >> 172.21.15.133:0/204263095 conn(0x7fa5240a2920 msgr2=0x7fa5240a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 -- 172.21.15.133:0/204263095 shutdown_connections 2024-04-02T11:26:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7fa529a24640 1 -- 172.21.15.133:0/204263095 wait complete. 2024-04-02T11:26:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.494+0000 7f225c8ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7fa529a24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f5d7c55d640 1 -- 172.21.15.133:0/2729492610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 msgr2=0x7f5d74136130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/2729492610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d74136130 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f79a3c2d640 1 -- 172.21.15.133:0/803754179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c131780 msgr2=0x7f799c131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/803754179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c131780 0x7f799c131b90 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f79a3c2d640 1 -- 172.21.15.133:0/803754179 shutdown_connections 2024-04-02T11:26:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/803754179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c131780 0x7f799c131b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f5d7c55d640 1 -- 172.21.15.133:0/2729492610 shutdown_connections 2024-04-02T11:26:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/2729492610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d74136130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f79a3c2d640 1 -- 172.21.15.133:0/803754179 >> 172.21.15.133:0/803754179 conn(0x7f799c0a4ba0 msgr2=0x7f799c0a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.495+0000 7f5d7c55d640 1 -- 172.21.15.133:0/2729492610 >> 172.21.15.133:0/2729492610 conn(0x7f5d740a0790 msgr2=0x7f5d740a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f79a3c2d640 1 -- 172.21.15.133:0/803754179 shutdown_connections 2024-04-02T11:26:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f5d7c55d640 1 -- 172.21.15.133:0/2729492610 shutdown_connections 2024-04-02T11:26:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f79a3c2d640 1 -- 172.21.15.133:0/803754179 wait complete. 2024-04-02T11:26:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7fa529a24640 1 Processor -- start 2024-04-02T11:26:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f79a3c2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f5d7c55d640 1 -- 172.21.15.133:0/2729492610 wait complete. 2024-04-02T11:26:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f5d7c55d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7fa529a24640 1 -- start start 2024-04-02T11:26:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f225c8ad640 1 Processor -- start 2024-04-02T11:26:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.496+0000 7f225c8ad640 1 -- start start 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa529a24640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa529a24640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5241207a0 con 0x7fa5240a4530 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa522ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa522ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35154/0 (socket says 172.21.15.133:35154) 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa522ffd640 1 -- 172.21.15.133:0/2429996237 learned_addr learned my addr 172.21.15.133:0/2429996237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7f79a3c2d640 1 Processor -- start 2024-04-02T11:26:57.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7f79a3c2d640 1 -- start start 2024-04-02T11:26:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.497+0000 7fa522ffd640 1 -- 172.21.15.133:0/2429996237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa524120fa0 con 0x7fa5240a4530 2024-04-02T11:26:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa522ffd640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa514014540 con 0x7fa5240a4530 2024-04-02T11:26:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa5140146a0 con 0x7fa5240a4530 2024-04-02T11:26:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52412f0f0 con 0x7fa5240a4530 2024-04-02T11:26:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7f225c8ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7f5d7c55d640 1 Processor -- start 2024-04-02T11:26:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7f5d7c55d640 1 -- start start 2024-04-02T11:26:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7f225c8ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22581c91a0 con 0x7f2258131780 2024-04-02T11:26:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa514014940 con 0x7fa5240a4530 2024-04-02T11:26:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52412ee20 con 0x7fa5240a4530 2024-04-02T11:26:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.498+0000 7f79a3c2d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f79a3c2d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f799c121080 con 0x7f799c09edf0 2024-04-02T11:26:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f225659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f79a19c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f79a19c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35174/0 (socket says 172.21.15.133:35174) 2024-04-02T11:26:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f79a19c9640 1 -- 172.21.15.133:0/3936007219 learned_addr learned my addr 172.21.15.133:0/3936007219 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f225659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35166/0 (socket says 172.21.15.133:35166) 2024-04-02T11:26:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f225659c640 1 -- 172.21.15.133:0/1434218930 learned_addr learned my addr 172.21.15.133:0/1434218930 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fa514014aa0 con 0x7fa5240a4530 2024-04-02T11:26:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7fa50bfff640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 0x7fa51003af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7fa514019040 con 0x7fa5240a4530 2024-04-02T11:26:57.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f79a19c9640 1 -- 172.21.15.133:0/3936007219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c11e410 con 0x7f799c09edf0 2024-04-02T11:26:57.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.499+0000 7f225659c640 1 -- 172.21.15.133:0/1434218930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225811f070 con 0x7f2258131780 2024-04-02T11:26:57.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f225659c640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7fa5227fc640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 0x7fa51003af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2248004c60 con 0x7f2258131780 2024-04-02T11:26:57.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f5d7c55d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa524063400 con 0x7fa5240a4530 2024-04-02T11:26:57.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2248004dc0 con 0x7f2258131780 2024-04-02T11:26:57.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2248007250 con 0x7f2258131780 2024-04-02T11:26:57.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f5d7c55d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7411dbc0 con 0x7f5d7406d890 2024-04-02T11:26:57.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79a19c9640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225811f6b0 con 0x7f2258131780 2024-04-02T11:26:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2258072770 con 0x7f2258131780 2024-04-02T11:26:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f5d7a2f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f799800ee00 con 0x7f799c09edf0 2024-04-02T11:26:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7998015420 con 0x7f799c09edf0 2024-04-02T11:26:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f79980156c0 con 0x7f799c09edf0 2024-04-02T11:26:57.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f5d7a2f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35190/0 (socket says 172.21.15.133:35190) 2024-04-02T11:26:57.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c11f400 con 0x7f799c09edf0 2024-04-02T11:26:57.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f5d7a2f9640 1 -- 172.21.15.133:0/3739190394 learned_addr learned my addr 172.21.15.133:0/3739190394 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.500+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c11fc00 con 0x7f799c09edf0 2024-04-02T11:26:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.501+0000 7f5d7a2f9640 1 -- 172.21.15.133:0/3739190394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d741205e0 con 0x7f5d7406d890 2024-04-02T11:26:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.501+0000 7fa5227fc640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 0x7fa51003af30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.501+0000 7f5d7a2f9640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.501+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258063400 con 0x7f2258131780 2024-04-02T11:26:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.502+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c063400 con 0x7f799c09edf0 2024-04-02T11:26:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.506+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fa514018190 con 0x7fa5240a4530 2024-04-02T11:26:57.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d70005a60 con 0x7f5d7406d890 2024-04-02T11:26:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f2248007400 con 0x7f2258131780 2024-04-02T11:26:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d70005bc0 con 0x7f5d7406d890 2024-04-02T11:26:57.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d74120810 con 0x7f5d7406d890 2024-04-02T11:26:57.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f224effd640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 0x7f223803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d70005e60 con 0x7f5d7406d890 2024-04-02T11:26:57.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7412dd10 con 0x7f5d7406d890 2024-04-02T11:26:57.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f2248048490 con 0x7f2258131780 2024-04-02T11:26:57.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f2255d9b640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 0x7f223803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f7998015870 con 0x7f799c09edf0 2024-04-02T11:26:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.507+0000 7f79927fc640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 0x7f798003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.508+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f79980499e0 con 0x7f799c09edf0 2024-04-02T11:26:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.508+0000 7f79a11c8640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 0x7f798003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.508+0000 7f2255d9b640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 0x7f223803ae90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.508+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7412dea0 con 0x7f5d7406d890 2024-04-02T11:26:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.508+0000 7f79a11c8640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 0x7f798003ae90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.512+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f224800ddf0 con 0x7f2258131780 2024-04-02T11:26:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.517+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f7998012c30 con 0x7f799c09edf0 2024-04-02T11:26:57.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.519+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f5d700070b0 con 0x7f5d7406d890 2024-04-02T11:26:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.519+0000 7f5d6affd640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 0x7f5d5803af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.519+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f5d700484b0 con 0x7f5d7406d890 2024-04-02T11:26:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.519+0000 7f5d79af8640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 0x7f5d5803af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.520+0000 7f5d79af8640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 0x7f5d5803af30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.523+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f5d7000a4e0 con 0x7f5d7406d890 2024-04-02T11:26:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.697+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f22581322a0 con 0x7f2258131780 2024-04-02T11:26:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.698+0000 7f224effd640 1 -- 172.21.15.133:0/1434218930 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f22581322a0 con 0x7f2258131780 2024-04-02T11:26:57.698 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 msgr2=0x7f223803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 0x7f223803ae90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 msgr2=0x7f22581c9660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 shutdown_connections 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2238038980 0x7f223803ae90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 --2- 172.21.15.133:0/1434218930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2258131780 0x7f22581c9660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 >> 172.21.15.133:0/1434218930 conn(0x7f22580a4ba0 msgr2=0x7f22581bca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 shutdown_connections 2024-04-02T11:26:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.701+0000 7f225c8ad640 1 -- 172.21.15.133:0/1434218930 wait complete. 2024-04-02T11:26:57.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.2 2024-04-02T11:26:57.717 DEBUG:teuthology.parallel:result is None 2024-04-02T11:26:57.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.718+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5d74063400 con 0x7f5d7406d890 2024-04-02T11:26:57.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.718+0000 7f5d6affd640 1 -- 172.21.15.133:0/3739190394 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f5d7000a120 con 0x7f5d7406d890 2024-04-02T11:26:57.719 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 msgr2=0x7f5d5803af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 0x7f5d5803af30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 msgr2=0x7f5d7411cb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 shutdown_connections 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d58038a20 0x7f5d5803af30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 --2- 172.21.15.133:0/3739190394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d7406d890 0x7f5d7411cb20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 >> 172.21.15.133:0/3739190394 conn(0x7f5d740a0790 msgr2=0x7f5d7412ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.721+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 shutdown_connections 2024-04-02T11:26:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.722+0000 7f5d7c55d640 1 -- 172.21.15.133:0/3739190394 wait complete. 2024-04-02T11:26:57.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.723+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa524068910 con 0x7fa5240a4530 2024-04-02T11:26:57.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.723+0000 7fa50bfff640 1 -- 172.21.15.133:0/2429996237 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fa51400f2a0 con 0x7fa5240a4530 2024-04-02T11:26:57.724 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 msgr2=0x7fa51003af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 0x7fa51003af30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 msgr2=0x7fa52406e4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 shutdown_connections 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fa510038a20 0x7fa51003af30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 --2- 172.21.15.133:0/2429996237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5240a4530 0x7fa52406e4a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 >> 172.21.15.133:0/2429996237 conn(0x7fa5240a2920 msgr2=0x7fa5241b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 shutdown_connections 2024-04-02T11:26:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.726+0000 7fa529a24640 1 -- 172.21.15.133:0/2429996237 wait complete. 2024-04-02T11:26:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.733+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f799c11f770 con 0x7f799c09edf0 2024-04-02T11:26:57.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.734+0000 7f79927fc640 1 -- 172.21.15.133:0/3936007219 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f7998000ed0 con 0x7f799c09edf0 2024-04-02T11:26:57.734 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-04-02T11:26:57.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.1 2024-04-02T11:26:57.735 DEBUG:teuthology.parallel:result is None 2024-04-02T11:26:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 msgr2=0x7f798003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 0x7f798003ae90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 msgr2=0x7f799c1201f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 shutdown_connections 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7980038980 0x7f798003ae90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 --2- 172.21.15.133:0/3936007219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f799c09edf0 0x7f799c1201f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 >> 172.21.15.133:0/3936007219 conn(0x7f799c0a4ba0 msgr2=0x7f799c133640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 shutdown_connections 2024-04-02T11:26:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.736+0000 7f79a3c2d640 1 -- 172.21.15.133:0/3936007219 wait complete. 2024-04-02T11:26:57.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.3 2024-04-02T11:26:57.738 DEBUG:teuthology.parallel:result is None 2024-04-02T11:26:57.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-04-02T11:26:57.747 DEBUG:teuthology.parallel:result is None 2024-04-02T11:26:57.747 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-02T11:26:57.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T11:26:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.893+0000 7f750cc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.898+0000 7f750cc6d640 1 Processor -- start 2024-04-02T11:26:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.898+0000 7f750cc6d640 1 -- start start 2024-04-02T11:26:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.899+0000 7f750cc6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f750809a9e0 0x7f750809adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.899+0000 7f750cc6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750809bdf0 con 0x7f750809b330 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.899+0000 7f750cc6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f750809c020 con 0x7f750809a9e0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f750809a9e0 0x7f750809adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f7505d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f750809b330 0x7f750812e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50944/0 (socket says 172.21.15.133:50944) 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f7505d9b640 1 -- 172.21.15.133:0/1618074372 learned_addr learned my addr 172.21.15.133:0/1618074372 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750659c640 1 -- 172.21.15.133:0/1618074372 >> v1:172.21.15.133:6789/0 conn(0x7f750809b330 legacy=0x7f750812e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750659c640 1 -- 172.21.15.133:0/1618074372 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750809d670 con 0x7f750809a9e0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750659c640 1 --2- 172.21.15.133:0/1618074372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f750809adf0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a1c6d67da20c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750559a640 1 -- 172.21.15.133:0/1618074372 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74f800e040 con 0x7f750809a9e0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750559a640 1 -- 172.21.15.133:0/1618074372 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f74f8007670 con 0x7f750809a9e0 2024-04-02T11:26:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.900+0000 7f750559a640 1 -- 172.21.15.133:0/1618074372 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74f8007910 con 0x7f750809a9e0 2024-04-02T11:26:57.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 -- 172.21.15.133:0/1618074372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 msgr2=0x7f750809adf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:57.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 --2- 172.21.15.133:0/1618074372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f750809adf0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 -- 172.21.15.133:0/1618074372 shutdown_connections 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 --2- 172.21.15.133:0/1618074372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f750809adf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 -- 172.21.15.133:0/1618074372 >> 172.21.15.133:0/1618074372 conn(0x7f75080a18a0 msgr2=0x7f75080a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 -- 172.21.15.133:0/1618074372 shutdown_connections 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 1 -- 172.21.15.133:0/1618074372 wait complete. 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.901+0000 7f750cc6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.903+0000 7f750cc6d640 1 Processor -- start 2024-04-02T11:26:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.903+0000 7f750cc6d640 1 -- start start 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750cc6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750cc6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f750811e340 con 0x7f750809a9e0 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35214/0 (socket says 172.21.15.133:35214) 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750659c640 1 -- 172.21.15.133:0/2657362851 learned_addr learned my addr 172.21.15.133:0/2657362851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750659c640 1 -- 172.21.15.133:0/2657362851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750811e8b0 con 0x7f750809a9e0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.904+0000 7f750659c640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.905+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74f800e040 con 0x7f750809a9e0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.905+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f74f8012a80 con 0x7f750809a9e0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.905+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750811eae0 con 0x7f750809a9e0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.905+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f74f8012d20 con 0x7f750809a9e0 2024-04-02T11:26:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.905+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750811fdf0 con 0x7f750809a9e0 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.906+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7508063400 con 0x7f750809a9e0 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.906+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f74f80153f0 con 0x7f750809a9e0 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.906+0000 7f74f6ffd640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 0x7f74e803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.906+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f74f80498b0 con 0x7f750809a9e0 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.907+0000 7f7505d9b640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 0x7f74e803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.907+0000 7f7505d9b640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 0x7f74e803ae90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:57.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:57.911+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f74f800fc80 con 0x7f750809a9e0 2024-04-02T11:26:58.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.083+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f750809adf0 con 0x7f74e8038980 2024-04-02T11:26:58.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.084+0000 7f74f6ffd640 1 -- 172.21.15.133:0/2657362851 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31069 (crc 0 0 0) 0x7f750809adf0 con 0x7f74e8038980 2024-04-02T11:26:58.085 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T11:26:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 msgr2=0x7f74e803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 0x7f74e803ae90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 msgr2=0x7f75081c7700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 shutdown_connections 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f74e8038980 0x7f74e803ae90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 --2- 172.21.15.133:0/2657362851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f750809a9e0 0x7f75081c7700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 >> 172.21.15.133:0/2657362851 conn(0x7f75080a18a0 msgr2=0x7f75081b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 shutdown_connections 2024-04-02T11:26:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.087+0000 7f750cc6d640 1 -- 172.21.15.133:0/2657362851 wait complete. 2024-04-02T11:26:58.098 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-02T11:26:57.307691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109748,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320652,"statfs":{"total":429496729600,"available":429384347648,"internally_reserved":0,"allocated":1662976,"data_stored":1299808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.194085+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:26:56.194085+0000","last_peered":"2024-04-02T11:26:56.194085+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:26:56.194085+0000","last_undegraded":"2024-04-02T11:26:56.194085+0000","last_fullsized":"2024-04-02T11:26:56.194085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-02T11:26:57.128346+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:26:57.128346+0000","last_peered":"2024-04-02T11:26:57.128346+0000","last_clean":"2024-04-02T11:26:57.128346+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:26:57.128346+0000","last_undegraded":"2024-04-02T11:26:57.128346+0000","last_fullsized":"2024-04-02T11:26:57.128346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.179050+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:26:56.179050+0000","last_peered":"2024-04-02T11:26:56.179050+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:26:56.179050+0000","last_undegraded":"2024-04-02T11:26:56.179050+0000","last_fullsized":"2024-04-02T11:26:56.179050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-02T11:26:57.125057+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:26:57.125057+0000","last_peered":"2024-04-02T11:26:57.125057+0000","last_clean":"2024-04-02T11:26:57.125057+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:26:57.125057+0000","last_undegraded":"2024-04-02T11:26:57.125057+0000","last_fullsized":"2024-04-02T11:26:57.125057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.177246+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:26:56.177246+0000","last_peered":"2024-04-02T11:26:56.177246+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:26:56.177246+0000","last_undegraded":"2024-04-02T11:26:56.177246+0000","last_fullsized":"2024-04-02T11:26:56.177246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.165646+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:26:56.165646+0000","last_peered":"2024-04-02T11:26:56.165646+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:26:56.165646+0000","last_undegraded":"2024-04-02T11:26:56.165646+0000","last_fullsized":"2024-04-02T11:26:56.165646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.177196+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:26:56.177196+0000","last_peered":"2024-04-02T11:26:56.177196+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:26:56.177196+0000","last_undegraded":"2024-04-02T11:26:56.177196+0000","last_fullsized":"2024-04-02T11:26:56.177196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.178072+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:26:56.178072+0000","last_peered":"2024-04-02T11:26:56.178072+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:26:56.178072+0000","last_undegraded":"2024-04-02T11:26:56.178072+0000","last_fullsized":"2024-04-02T11:26:56.178072+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.124496+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:26:56.124496+0000","last_peered":"2024-04-02T11:26:56.124496+0000","last_clean":"2024-04-02T11:26:56.124496+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:26:56.124496+0000","last_undegraded":"2024-04-02T11:26:56.124496+0000","last_fullsized":"2024-04-02T11:26:56.124496+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":712704,"data_stored":620136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T11:26:58.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T11:26:58.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.205+0000 7fb32b86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:58.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.209+0000 7fb32b86d640 1 Processor -- start 2024-04-02T11:26:58.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.210+0000 7fb32b86d640 1 -- start start 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb32b86d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb324131f50 0x7fb324136600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb32b86d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb32406d300 con 0x7fb32409c820 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb32b86d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fb32409cce0 con 0x7fb324131f50 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb328e08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb324131f50 0x7fb324136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb329609640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb32409c820 0x7fb324131840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50948/0 (socket says 172.21.15.133:50948) 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb329609640 1 -- 172.21.15.133:0/3564468931 learned_addr learned my addr 172.21.15.133:0/3564468931 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb328e08640 1 -- 172.21.15.133:0/3564468931 >> v1:172.21.15.133:6789/0 conn(0x7fb32409c820 legacy=0x7fb324131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb328e08640 1 -- 172.21.15.133:0/3564468931 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32409c380 con 0x7fb324131f50 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb31bfff640 1 -- 172.21.15.133:0/3564468931 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622821546 0 0) 0x7fb32406d300 con 0x7fb32409c820 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb328e08640 1 --2- 172.21.15.133:0/3564468931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb324136600 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b19454723fe6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb31bfff640 1 -- 172.21.15.133:0/3564468931 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb314009d90 con 0x7fb324131f50 2024-04-02T11:26:58.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb31bfff640 1 -- 172.21.15.133:0/3564468931 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb31400e230 con 0x7fb324131f50 2024-04-02T11:26:58.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.211+0000 7fb31bfff640 1 -- 172.21.15.133:0/3564468931 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb31400e4d0 con 0x7fb324131f50 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 -- 172.21.15.133:0/3564468931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 msgr2=0x7fb324136600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 --2- 172.21.15.133:0/3564468931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb324136600 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 -- 172.21.15.133:0/3564468931 shutdown_connections 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 --2- 172.21.15.133:0/3564468931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb324136600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 -- 172.21.15.133:0/3564468931 >> 172.21.15.133:0/3564468931 conn(0x7fb3240a0790 msgr2=0x7fb3240a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 -- 172.21.15.133:0/3564468931 shutdown_connections 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.212+0000 7fb32b86d640 1 -- 172.21.15.133:0/3564468931 wait complete. 2024-04-02T11:26:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.213+0000 7fb32b86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.214+0000 7fb32b86d640 1 Processor -- start 2024-04-02T11:26:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.214+0000 7fb32b86d640 1 -- start start 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb32b86d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb32b86d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3240721a0 con 0x7fb324131f50 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb329609640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb329609640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35238/0 (socket says 172.21.15.133:35238) 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb329609640 1 -- 172.21.15.133:0/170314963 learned_addr learned my addr 172.21.15.133:0/170314963 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb329609640 1 -- 172.21.15.133:0/170314963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb324135b40 con 0x7fb324131f50 2024-04-02T11:26:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb329609640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb32000ec00 con 0x7fb324131f50 2024-04-02T11:26:58.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb32000ed60 con 0x7fb324131f50 2024-04-02T11:26:58.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.215+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb324072390 con 0x7fb324131f50 2024-04-02T11:26:58.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.216+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb320004840 con 0x7fb324131f50 2024-04-02T11:26:58.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.216+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb324134f10 con 0x7fb324131f50 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.216+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fb3200049f0 con 0x7fb324131f50 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.217+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb324136370 con 0x7fb324131f50 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.217+0000 7fb319ffb640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 0x7fb30803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.217+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7fb32004ac70 con 0x7fb324131f50 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.217+0000 7fb328e08640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 0x7fb30803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.218+0000 7fb328e08640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 0x7fb30803ae90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.222+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fb3200103d0 con 0x7fb324131f50 2024-04-02T11:26:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.396+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb324136b40 con 0x7fb308038980 2024-04-02T11:26:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.398+0000 7fb319ffb640 1 -- 172.21.15.133:0/170314963 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31069 (crc 0 0 0) 0x7fb314009e00 con 0x7fb308038980 2024-04-02T11:26:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:58.399 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 msgr2=0x7fb30803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 0x7fb30803ae90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 msgr2=0x7fb3241325e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 shutdown_connections 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fb308038980 0x7fb30803ae90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 --2- 172.21.15.133:0/170314963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb324131f50 0x7fb3241325e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 >> 172.21.15.133:0/170314963 conn(0x7fb3240a0790 msgr2=0x7fb32409d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 shutdown_connections 2024-04-02T11:26:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.400+0000 7fb32b86d640 1 -- 172.21.15.133:0/170314963 wait complete. 2024-04-02T11:26:58.412 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-02T11:26:57.307691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109748,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320652,"statfs":{"total":429496729600,"available":429384347648,"internally_reserved":0,"allocated":1662976,"data_stored":1299808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.194085+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:26:56.194085+0000","last_peered":"2024-04-02T11:26:56.194085+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:26:56.194085+0000","last_undegraded":"2024-04-02T11:26:56.194085+0000","last_fullsized":"2024-04-02T11:26:56.194085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-02T11:26:57.128346+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:26:57.128346+0000","last_peered":"2024-04-02T11:26:57.128346+0000","last_clean":"2024-04-02T11:26:57.128346+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:26:57.128346+0000","last_undegraded":"2024-04-02T11:26:57.128346+0000","last_fullsized":"2024-04-02T11:26:57.128346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.179050+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:26:56.179050+0000","last_peered":"2024-04-02T11:26:56.179050+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:26:56.179050+0000","last_undegraded":"2024-04-02T11:26:56.179050+0000","last_fullsized":"2024-04-02T11:26:56.179050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-02T11:26:57.125057+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:26:57.125057+0000","last_peered":"2024-04-02T11:26:57.125057+0000","last_clean":"2024-04-02T11:26:57.125057+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:26:57.125057+0000","last_undegraded":"2024-04-02T11:26:57.125057+0000","last_fullsized":"2024-04-02T11:26:57.125057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.177246+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:26:56.177246+0000","last_peered":"2024-04-02T11:26:56.177246+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:26:56.177246+0000","last_undegraded":"2024-04-02T11:26:56.177246+0000","last_fullsized":"2024-04-02T11:26:56.177246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.165646+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:26:56.165646+0000","last_peered":"2024-04-02T11:26:56.165646+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:26:56.165646+0000","last_undegraded":"2024-04-02T11:26:56.165646+0000","last_fullsized":"2024-04-02T11:26:56.165646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.177196+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:26:56.177196+0000","last_peered":"2024-04-02T11:26:56.177196+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:26:56.177196+0000","last_undegraded":"2024-04-02T11:26:56.177196+0000","last_fullsized":"2024-04-02T11:26:56.177196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.178072+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:26:56.178072+0000","last_peered":"2024-04-02T11:26:56.178072+0000","last_clean":"2024-04-02T11:26:55.099376+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:26:56.178072+0000","last_undegraded":"2024-04-02T11:26:56.178072+0000","last_fullsized":"2024-04-02T11:26:56.178072+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-02T11:26:56.124496+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:26:56.124496+0000","last_peered":"2024-04-02T11:26:56.124496+0000","last_clean":"2024-04-02T11:26:56.124496+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:26:56.124496+0000","last_undegraded":"2024-04-02T11:26:56.124496+0000","last_fullsized":"2024-04-02T11:26:56.124496+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":712704,"data_stored":620136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T11:26:58.413 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-02T11:26:58.413 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T11:26:58.413 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-02T11:26:58.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-02T11:26:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.519+0000 7f326a5e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.523+0000 7f326a5e0640 1 Processor -- start 2024-04-02T11:26:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.523+0000 7f326a5e0640 1 -- start start 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f326a5e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3264137950 0x7f3264139d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f326a5e0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32641322a0 con 0x7f3264131780 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f326a5e0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f32640a6af0 con 0x7f3264137950 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f32637fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3264137950 0x7f3264139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f3263fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3264131780 0x7f3264131b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50952/0 (socket says 172.21.15.133:50952) 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f3263fff640 1 -- 172.21.15.133:0/3421661618 learned_addr learned my addr 172.21.15.133:0/3421661618 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f32637fe640 1 -- 172.21.15.133:0/3421661618 >> v1:172.21.15.133:6789/0 conn(0x7f3264131780 legacy=0x7f3264131b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f32637fe640 1 -- 172.21.15.133:0/3421661618 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3264073660 con 0x7f3264137950 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.524+0000 7f32637fe640 1 --2- 172.21.15.133:0/3421661618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264137950 0x7f3264139d70 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0c70654d1f3df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f3262ffd640 1 -- 172.21.15.133:0/3421661618 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3258009d90 con 0x7f3264137950 2024-04-02T11:26:58.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f3262ffd640 1 -- 172.21.15.133:0/3421661618 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f325800e230 con 0x7f3264137950 2024-04-02T11:26:58.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f3262ffd640 1 -- 172.21.15.133:0/3421661618 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f325800e4d0 con 0x7f3264137950 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 -- 172.21.15.133:0/3421661618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264137950 msgr2=0x7f3264139d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 --2- 172.21.15.133:0/3421661618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264137950 0x7f3264139d70 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 -- 172.21.15.133:0/3421661618 shutdown_connections 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 --2- 172.21.15.133:0/3421661618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264137950 0x7f3264139d70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 -- 172.21.15.133:0/3421661618 >> 172.21.15.133:0/3421661618 conn(0x7f32640a4ba0 msgr2=0x7f32640a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 -- 172.21.15.133:0/3421661618 shutdown_connections 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.525+0000 7f326a5e0640 1 -- 172.21.15.133:0/3421661618 wait complete. 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.526+0000 7f326a5e0640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:26:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.527+0000 7f326a5e0640 1 Processor -- start 2024-04-02T11:26:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.527+0000 7f326a5e0640 1 -- start start 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f326a5e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f326a5e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f326411f820 con 0x7f3264131780 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f3263fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f3263fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35258/0 (socket says 172.21.15.133:35258) 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f3263fff640 1 -- 172.21.15.133:0/1340516410 learned_addr learned my addr 172.21.15.133:0/1340516410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.528+0000 7f3263fff640 1 -- 172.21.15.133:0/1340516410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3264131050 con 0x7f3264131780 2024-04-02T11:26:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f3263fff640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3254005a60 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3254005bc0 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3264121080 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3254005e60 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.529+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326411fc70 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.530+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f325400c780 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.530+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3264131240 con 0x7f3264131780 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.530+0000 7f3260ff9640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 0x7f325003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:26:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.530+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2672+0+0 (crc 0 0 0) 0x7f3254048030 con 0x7f3264131780 2024-04-02T11:26:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.530+0000 7f32637fe640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 0x7f325003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:26:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.531+0000 7f32637fe640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 0x7f325003ae90 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:26:58.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.535+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f325400d150 con 0x7f3264131780 2024-04-02T11:26:58.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.748+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f32641322a0 con 0x7f3264131780 2024-04-02T11:26:58.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.748+0000 7f3260ff9640 1 -- 172.21.15.133:0/1340516410 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (crc 0 0 0) 0x7f32641322a0 con 0x7f3264131780 2024-04-02T11:26:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T11:26:58.749 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T11:26:58.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 msgr2=0x7f325003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 0x7f325003ae90 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 msgr2=0x7f326411d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:26:58.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 shutdown_connections 2024-04-02T11:26:58.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3250038980 0x7f325003ae90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 --2- 172.21.15.133:0/1340516410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264131780 0x7f326411d2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:26:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.750+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 >> 172.21.15.133:0/1340516410 conn(0x7f32640a4ba0 msgr2=0x7f32641bca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:26:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.751+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 shutdown_connections 2024-04-02T11:26:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T11:26:58.751+0000 7f326a5e0640 1 -- 172.21.15.133:0/1340516410 wait complete. 2024-04-02T11:26:58.762 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-02T11:26:58.762 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T11:26:58.779 INFO:tasks.workunit:Pulling workunits from ref 107e1624f2a9b72a56ee529969558944d24a2a75 2024-04-02T11:26:58.779 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T11:26:58.779 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T11:26:58.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T11:26:58.794 INFO:teuthology.orchestra.run.smithi133.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-02T11:26:58.799 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-02T11:26:58.849 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-02T11:26:58.849 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-02T11:26:58.904 INFO:tasks.workunit:timeout=3h 2024-04-02T11:26:58.904 INFO:tasks.workunit:cleanup=True 2024-04-02T11:26:58.904 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 107e1624f2a9b72a56ee529969558944d24a2a75 2024-04-02T11:26:58.960 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T11:27:58.494 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to '107e1624f2a9b72a56ee529969558944d24a2a75'. 2024-04-02T11:27:58.497 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.497 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T11:27:58.497 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T11:27:58.497 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T11:27:58.497 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-04-02T11:27:58.498 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at 107e1624f2a Merge pull request #53679 from rhcs-dashboard/alerts-v2-api 2024-04-02T11:27:58.499 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T11:27:58.559 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T11:27:58.561 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T11:27:58.562 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T11:27:59.325 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T11:27:59.383 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T11:27:59.424 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T11:27:59.425 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T11:27:59.425 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T11:27:59.464 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T11:27:59.467 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-02T11:27:59.467 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T11:27:59.523 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-04-02T11:27:59.524 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-04-02T11:27:59.525 DEBUG:teuthology.orchestra.run.smithi133:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=107e1624f2a9b72a56ee529969558944d24a2a75 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-04-02T11:27:59.583 INFO:tasks.workunit.client.0.smithi133.stderr:+ parallel=1 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' --crimson = --serial ']' 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ crimson=0 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' --crimson = --crimson ']' 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ parallel=0 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ crimson=1 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ color= 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' -t 1 ']' 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-04-02T11:27:59.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-02T11:27:59.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-02T11:27:59.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ declare -A pids 2024-04-02T11:27:59.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:27:59.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:27:59.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_aio 2024-04-02T11:27:59.601 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:27:59.601 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 42 tests from 2 test suites. 2024-04-02T11:27:59.601 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:27:59.601 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 26 tests from LibRadosAio 2024-04-02T11:27:59.601 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.TooBig 2024-04-02T11:28:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.TooBig (2574 ms) 2024-04-02T11:28:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-04-02T11:28:05.191 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleWrite (3015 ms) 2024-04-02T11:28:05.191 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-04-02T11:28:08.201 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.WaitForSafe (3011 ms) 2024-04-02T11:28:08.201 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-04-02T11:28:11.221 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTrip (3020 ms) 2024-04-02T11:28:11.221 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-04-02T11:28:14.232 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTrip2 (3011 ms) 2024-04-02T11:28:14.232 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-04-02T11:28:17.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTrip3 (3016 ms) 2024-04-02T11:28:17.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-04-02T11:28:20.259 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripAppend (3010 ms) 2024-04-02T11:28:20.259 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-04-02T11:28:23.276 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RemoveTest (3017 ms) 2024-04-02T11:28:23.276 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-04-02T11:28:26.285 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3009 ms) 2024-04-02T11:28:26.285 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RmXattr 2024-04-02T11:28:29.304 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RmXattr (3020 ms) 2024-04-02T11:28:29.305 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.XattrIter 2024-04-02T11:28:32.317 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.XattrIter (3013 ms) 2024-04-02T11:28:32.317 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.IsComplete 2024-04-02T11:28:35.330 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.IsComplete (3013 ms) 2024-04-02T11:28:35.330 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.IsSafe 2024-04-02T11:28:38.339 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.IsSafe (3008 ms) 2024-04-02T11:28:38.339 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-04-02T11:28:41.358 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.ReturnValue (3019 ms) 2024-04-02T11:28:41.358 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.Flush 2024-04-02T11:28:44.367 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.Flush (3009 ms) 2024-04-02T11:28:44.367 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-04-02T11:28:47.385 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.FlushAsync (3018 ms) 2024-04-02T11:28:47.386 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-04-02T11:28:50.395 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3010 ms) 2024-04-02T11:28:50.395 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-04-02T11:28:53.411 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (3015 ms) 2024-04-02T11:28:53.411 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-04-02T11:28:55.994 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleStat (2583 ms) 2024-04-02T11:28:55.994 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-04-02T11:28:59.007 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.OperateMtime (3013 ms) 2024-04-02T11:28:59.007 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-04-02T11:29:02.020 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.Operate2Mtime (3014 ms) 2024-04-02T11:29:02.020 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-04-02T11:29:05.037 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleStatNS (3017 ms) 2024-04-02T11:29:05.037 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.StatRemove 2024-04-02T11:29:08.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.StatRemove (3018 ms) 2024-04-02T11:29:08.056 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-04-02T11:29:11.070 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.ExecuteClass (3012 ms) 2024-04-02T11:29:11.070 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-04-02T11:29:13.994 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.MultiWrite (2927 ms) 2024-04-02T11:29:13.994 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.AioUnlock (3016 ms) 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 26 tests from LibRadosAio (77409 ms total) 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 16 tests from LibRadosAioEC 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-04-02T11:29:17.012 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1100: Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1132: Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1149: Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1182: Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-04-02T11:29:17.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1215: Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1286: Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1326: Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1366: Skipped 2024-04-02T11:29:17.014 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.Flush 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1385: Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1415: Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1454: Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-04-02T11:29:17.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1498: Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1532: Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1593: Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1649: Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.016 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio.cc:1681: Skipped 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 42 tests from 2 test suites ran. (77409 ms total) 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 26 tests. 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 16 tests, listed below: 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-04-02T11:29:17.017 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-04-02T11:29:17.018 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-04-02T11:29:17.019 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:29:17.019 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:29:17.019 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_aio_pp 2024-04-02T11:29:17.032 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:29:17.032 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 57 tests from 4 test suites. 2024-04-02T11:29:17.032 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:29:17.032 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 32 tests from LibRadosAio 2024-04-02T11:29:17.032 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-04-02T11:29:20.024 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.TooBigPP (2992 ms) 2024-04-02T11:29:20.024 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-04-02T11:29:37.014 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.PoolQuotaPP (16990 ms) 2024-04-02T11:29:37.015 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-04-02T11:29:43.048 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleWritePP (6034 ms) 2024-04-02T11:29:43.048 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-04-02T11:29:46.062 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.WaitForSafePP (3013 ms) 2024-04-02T11:29:46.062 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-04-02T11:29:49.004 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripPP (2942 ms) 2024-04-02T11:29:49.004 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-04-02T11:29:52.082 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3079 ms) 2024-04-02T11:29:52.083 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-04-02T11:29:55.107 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3024 ms) 2024-04-02T11:29:55.107 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-04-02T11:29:58.116 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3010 ms) 2024-04-02T11:29:58.116 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-04-02T11:30:01.124 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.IsCompletePP (3008 ms) 2024-04-02T11:30:01.124 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-04-02T11:30:04.137 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.IsSafePP (3013 ms) 2024-04-02T11:30:04.137 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-04-02T11:30:07.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.ReturnValuePP (3012 ms) 2024-04-02T11:30:07.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.FlushPP 2024-04-02T11:30:10.164 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.FlushPP (3014 ms) 2024-04-02T11:30:10.164 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-04-02T11:30:13.181 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3017 ms) 2024-04-02T11:30:13.181 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-04-02T11:30:16.194 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3014 ms) 2024-04-02T11:30:16.195 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-04-02T11:30:19.007 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2812 ms) 2024-04-02T11:30:19.007 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-04-02T11:30:22.010 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3004 ms) 2024-04-02T11:30:22.010 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-04-02T11:30:25.031 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3020 ms) 2024-04-02T11:30:25.031 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-04-02T11:30:28.041 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3011 ms) 2024-04-02T11:30:28.042 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-04-02T11:30:31.054 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.SimpleStatPP (3013 ms) 2024-04-02T11:30:31.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-04-02T11:30:34.068 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.OperateMtime (3014 ms) 2024-04-02T11:30:34.068 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-04-02T11:30:37.083 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.OperateMtime2 (3015 ms) 2024-04-02T11:30:37.083 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-04-02T11:30:40.096 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-04-02T11:30:40.096 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-04-02T11:30:43.110 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3015 ms) 2024-04-02T11:30:43.111 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.OmapPP 2024-04-02T11:30:46.119 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.OmapPP (3009 ms) 2024-04-02T11:30:46.119 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-04-02T11:30:49.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.MultiWritePP (2893 ms) 2024-04-02T11:30:49.013 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-04-02T11:30:52.148 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.AioUnlockPP (3136 ms) 2024-04-02T11:30:52.148 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-04-02T11:30:55.159 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3011 ms) 2024-04-02T11:30:55.159 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-04-02T11:30:58.173 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RacingRemovePP (3014 ms) 2024-04-02T11:30:58.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-04-02T11:31:01.182 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3009 ms) 2024-04-02T11:31:01.182 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-04-02T11:31:04.195 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3013 ms) 2024-04-02T11:31:04.195 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-04-02T11:31:06.706 INFO:tasks.workunit.client.0.smithi133.stdout:setting pool EIO 2024-04-02T11:31:16.374 INFO:tasks.workunit.client.0.smithi133.stdout:waiting for 815 2024-04-02T11:31:17.374 INFO:tasks.workunit.client.0.smithi133.stdout:waiting for 815 2024-04-02T11:31:18.374 INFO:tasks.workunit.client.0.smithi133.stdout:waiting for 815 2024-04-02T11:31:19.374 INFO:tasks.workunit.client.0.smithi133.stdout:max_success 185, min_failed 186 2024-04-02T11:31:20.223 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAio.PoolEIOFlag (16028 ms) 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAio.MultiReads 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:2427: Skipped 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 32 tests from LibRadosAio (123191 ms total) 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 4 tests from LibRadosAioPP 2024-04-02T11:31:20.224 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-04-02T11:31:23.237 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3014 ms) 2024-04-02T11:31:23.237 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-04-02T11:31:32.029 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8792 ms) 2024-04-02T11:31:32.030 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-04-02T11:31:47.101 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAioPP.RmXattrPP (15072 ms) 2024-04-02T11:31:47.101 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-04-02T11:31:50.116 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3015 ms) 2024-04-02T11:31:50.116 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 4 tests from LibRadosAioPP (29893 ms total) 2024-04-02T11:31:50.116 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:50.117 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosIoPP 2024-04-02T11:31:50.117 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-04-02T11:31:53.138 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.XattrListPP (3020 ms) 2024-04-02T11:31:53.138 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosIoPP (3020 ms total) 2024-04-02T11:31:53.138 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 20 tests from LibRadosAioEC 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1384: Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1419: Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1436: Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-04-02T11:31:53.139 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1467: Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1500: Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1605: Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1637: Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-04-02T11:31:53.140 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1705: Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1742: Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1780: Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1796: Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-04-02T11:31:53.141 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1824: Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1859: Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1904: Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1946: Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-04-02T11:31:53.142 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:1977: Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:2008: Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:2059: Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:2089: Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/aio_cxx.cc:2124: Skipped 2024-04-02T11:31:53.143 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 57 tests from 4 test suites ran. (156104 ms total) 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 36 tests. 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 21 tests, listed below: 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-04-02T11:31:53.144 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-04-02T11:31:53.145 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-04-02T11:31:53.146 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:31:53.146 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:31:53.146 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_io 2024-04-02T11:31:53.161 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:31:53.161 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 24 tests from 2 test suites. 2024-04-02T11:31:53.161 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:31:53.161 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 14 tests from LibRadosIo 2024-04-02T11:31:55.142 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-04-02T11:31:55.541 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.SimpleWrite (399 ms) 2024-04-02T11:31:55.541 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.TooBig 2024-04-02T11:31:55.541 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-04-02T11:31:55.541 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-04-02T11:31:55.610 INFO:tasks.workunit.client.0.smithi133.stdout:no timeout :/ 2024-04-02T11:31:55.610 INFO:tasks.workunit.client.0.smithi133.stdout:no timeout :/ 2024-04-02T11:31:55.610 INFO:tasks.workunit.client.0.smithi133.stdout:no timeout :/ 2024-04-02T11:31:55.610 INFO:tasks.workunit.client.0.smithi133.stdout:no timeout :/ 2024-04-02T11:31:55.611 INFO:tasks.workunit.client.0.smithi133.stdout:no timeout :/ 2024-04-02T11:31:55.611 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.ReadTimeout (69 ms) 2024-04-02T11:31:55.611 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-04-02T11:31:55.642 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.RoundTrip (31 ms) 2024-04-02T11:31:55.642 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.Checksum 2024-04-02T11:31:55.662 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.Checksum (21 ms) 2024-04-02T11:31:55.662 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-04-02T11:31:55.689 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (27 ms) 2024-04-02T11:31:55.689 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-04-02T11:31:55.714 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (25 ms) 2024-04-02T11:31:55.714 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-04-02T11:31:55.746 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.AppendRoundTrip (32 ms) 2024-04-02T11:31:55.746 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-04-02T11:31:55.751 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.ZeroLenZero (5 ms) 2024-04-02T11:31:55.751 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.TruncTest 2024-04-02T11:31:55.776 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.TruncTest (25 ms) 2024-04-02T11:31:55.776 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-04-02T11:31:55.800 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.RemoveTest (24 ms) 2024-04-02T11:31:55.800 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-04-02T11:31:55.830 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (30 ms) 2024-04-02T11:31:55.830 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.RmXattr 2024-04-02T11:31:55.905 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.RmXattr (75 ms) 2024-04-02T11:31:55.905 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIo.XattrIter 2024-04-02T11:31:55.943 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIo.XattrIter (38 ms) 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 14 tests from LibRadosIo (801 ms total) 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 10 tests from LibRadosIoEC 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:168: Skipped 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.148 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.149 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (1 ms) 2024-04-02T11:31:56.150 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.151 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:31:56.152 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:176: Skipped 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 10 tests from LibRadosIoEC (1 ms total) 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 24 tests from 2 test suites ran. (2985 ms total) 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 14 tests. 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 10 tests, listed below: 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-04-02T11:31:56.153 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:31:56.154 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_io_pp 2024-04-02T11:31:56.169 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:31:56.169 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 38 tests from 2 test suites. 2024-04-02T11:31:56.169 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:31:56.169 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 21 tests from LibRadosIoPP 2024-04-02T11:31:56.169 INFO:tasks.workunit.client.0.smithi133.stdout:seed 34158 2024-04-02T11:31:58.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-04-02T11:31:58.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-04-02T11:31:58.152 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-04-02T11:31:58.583 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (431 ms) 2024-04-02T11:31:58.584 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-04-02T11:31:58.651 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.ReadOpPP (68 ms) 2024-04-02T11:31:58.652 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-04-02T11:31:58.676 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (25 ms) 2024-04-02T11:31:58.677 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-04-02T11:32:01.218 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (2542 ms) 2024-04-02T11:32:01.218 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-04-02T11:32:01.238 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.RoundTripPP (20 ms) 2024-04-02T11:32:01.238 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-04-02T11:32:01.261 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (23 ms) 2024-04-02T11:32:01.261 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-04-02T11:32:01.279 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.Checksum (18 ms) 2024-04-02T11:32:01.279 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-04-02T11:32:01.295 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (16 ms) 2024-04-02T11:32:01.295 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-04-02T11:32:01.323 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (28 ms) 2024-04-02T11:32:01.323 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-04-02T11:32:01.349 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (26 ms) 2024-04-02T11:32:01.349 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-04-02T11:32:01.366 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (17 ms) 2024-04-02T11:32:01.366 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-04-02T11:32:01.393 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (27 ms) 2024-04-02T11:32:01.393 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-04-02T11:32:01.418 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.TruncTestPP (25 ms) 2024-04-02T11:32:01.419 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-04-02T11:32:01.443 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (24 ms) 2024-04-02T11:32:01.443 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-04-02T11:32:01.476 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (33 ms) 2024-04-02T11:32:01.476 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-04-02T11:32:01.556 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.RmXattrPP (80 ms) 2024-04-02T11:32:01.556 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-04-02T11:32:01.592 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.XattrListPP (36 ms) 2024-04-02T11:32:01.592 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-04-02T11:32:01.619 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.CmpExtPP (27 ms) 2024-04-02T11:32:01.619 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-04-02T11:32:01.636 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (17 ms) 2024-04-02T11:32:01.636 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-04-02T11:32:01.660 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (24 ms) 2024-04-02T11:32:02.169 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 21 tests from LibRadosIoPP (3508 ms total) 2024-04-02T11:32:02.169 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:02.169 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 17 tests from LibRadosIoECPP 2024-04-02T11:32:02.169 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-02T11:32:02.169 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-04-02T11:32:02.170 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-04-02T11:32:02.171 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-04-02T11:32:02.172 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-04-02T11:32:02.173 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.174 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-04-02T11:32:02.175 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.176 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 38 tests from 2 test suites ran. (5996 ms total) 2024-04-02T11:32:02.177 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 21 tests. 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 17 tests, listed below: 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-02T11:32:02.178 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:32:02.179 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_asio 2024-04-02T11:32:02.205 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.205+0000 7f6251a3e6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:02.211 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.211+0000 7f6251a3e6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:02.212 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.211+0000 7f6251a3e6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:02.212 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 12 tests from 1 test suite. 2024-04-02T11:32:02.212 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:32:02.212 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 12 tests from AsioRados 2024-04-02T11:32:02.212 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.212+0000 7f6251a3e6c0 1 Processor -- start 2024-04-02T11:32:02.213 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.212+0000 7f6251a3e6c0 1 -- start start 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.213+0000 7f6251a3e6c0 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.213+0000 7f6251a3e6c0 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x1546020 con 0x15ddbc0 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f624979a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f624979a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42700/0 (socket says 172.21.15.133:42700) 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f624979a640 1 -- 172.21.15.133:0/1974915064 learned_addr learned my addr 172.21.15.133:0/1974915064 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:32:02.214 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f624979a640 1 -- 172.21.15.133:0/1974915064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14e61e0 con 0x15ddbc0 2024-04-02T11:32:02.215 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f624979a640 1 --2- 172.21.15.133:0/1974915064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47560e1fa8511bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:32:02.215 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f623b7fe640 1 -- 172.21.15.133:0/1974915064 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f62400064b0 con 0x15ddbc0 2024-04-02T11:32:02.215 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f623b7fe640 1 -- 172.21.15.133:0/1974915064 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f624000a040 con 0x15ddbc0 2024-04-02T11:32:02.215 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.214+0000 7f623b7fe640 1 -- 172.21.15.133:0/1974915064 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f624000a2e0 con 0x15ddbc0 2024-04-02T11:32:02.216 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/1974915064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 msgr2=0x1704ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:32:02.216 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/1974915064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:02.216 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/1974915064 shutdown_connections 2024-04-02T11:32:02.216 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/1974915064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x1704ea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:02.217 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/1974915064 >> 172.21.15.133:0/1974915064 conn(0x14dd890 msgr2=0x1679840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:32:02.217 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/1974915064 shutdown_connections 2024-04-02T11:32:02.217 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.216+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/1974915064 wait complete. 2024-04-02T11:32:02.217 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.217+0000 7f6251a3e6c0 1 Processor -- start 2024-04-02T11:32:02.218 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.218+0000 7f6251a3e6c0 1 -- start start 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.219+0000 7f6251a3e6c0 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.219+0000 7f6251a3e6c0 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x1543aa0 con 0x15ddbc0 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.219+0000 7f624979a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.219+0000 7f624979a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42708/0 (socket says 172.21.15.133:42708) 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f624979a640 1 -- 172.21.15.133:0/3667714673 learned_addr learned my addr 172.21.15.133:0/3667714673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T11:32:02.220 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f624979a640 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14d8e30 con 0x15ddbc0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f624979a640 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f624003b040 con 0x15ddbc0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x14d8890 con 0x15ddbc0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f624000a2e0 con 0x15ddbc0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6240035580 con 0x15ddbc0 2024-04-02T11:32:02.221 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.220+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1703920 con 0x15ddbc0 2024-04-02T11:32:02.222 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.221+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f624003e040 con 0x15ddbc0 2024-04-02T11:32:02.222 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f623a7fc640 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 0x7f62280396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:32:02.222 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(235..235 src has 178..235) ==== 2672+0+0 (crc 0 0 0) 0x7f624002e040 con 0x15ddbc0 2024-04-02T11:32:02.223 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x153a130 con 0x15ddbc0 2024-04-02T11:32:02.223 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f6248f99640 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 0x7f62280396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T11:32:02.223 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=235) ==== 24+0+0 (crc 0 0 0) 0x7f624003d460 con 0x15ddbc0 2024-04-02T11:32:02.223 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.222+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x154b9a0 con 0x15ddbc0 2024-04-02T11:32:02.223 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:02.223+0000 7f6248f99640 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 0x7f62280396d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:32:03.169 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.168+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v236) ==== 43+0+0 (crc 0 0 0) 0x7f624003d640 con 0x15ddbc0 2024-04-02T11:32:03.169 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.168+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=236}) -- 0x7f622803bb40 con 0x15ddbc0 2024-04-02T11:32:03.169 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.169+0000 7f623a7fc640 1 -- 172.21.15.133:0/3667714673 <== mon.0 v2:172.21.15.133:3300/0 8 ==== osd_map(236..236 src has 178..236) ==== 651+0+0 (crc 0 0 0) 0x7f624003d850 con 0x15ddbc0 2024-04-02T11:32:03.169 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.169+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 0x17118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T11:32:03.169 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.169+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1711e00 con 0x16741d0 2024-04-02T11:32:03.173 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.173+0000 7f6249f9b640 1 --2- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 0x17118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T11:32:03.274 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:03.274+0000 7f6249f9b640 1 --2- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 0x17118c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T11:32:04.449 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.446+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 1 ==== osd_op_reply(2 exist [writefull 0~5] v237'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.450 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-04-02T11:32:04.450 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.446+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x170c580 con 0x16741d0 2024-04-02T11:32:04.450 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.446+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x170cdf0 con 0x16741d0 2024-04-02T11:32:04.464 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.464+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 2 ==== osd_op_reply(3 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.467 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.467+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 3 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.468 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadCallback (21 ms) 2024-04-02T11:32:04.468 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-04-02T11:32:04.468 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.467+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x170d1d0 con 0x16741d0 2024-04-02T11:32:04.468 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.467+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1712160 con 0x16741d0 2024-04-02T11:32:04.475 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.475+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.475 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.475+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.475 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadFuture (8 ms) 2024-04-02T11:32:04.475 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-04-02T11:32:04.476 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.475+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x170c920 con 0x16741d0 2024-04-02T11:32:04.476 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.475+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1753000 con 0x16741d0 2024-04-02T11:32:04.483 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.483+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 6 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.483 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.483+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v237'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.483 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadYield (8 ms) 2024-04-02T11:32:04.483 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-04-02T11:32:04.483 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.483+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x17134d0 con 0x16741d0 2024-04-02T11:32:04.496 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.495+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 8 ==== osd_op_reply(9 exist [write 0~5] v237'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.496 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteCallback (13 ms) 2024-04-02T11:32:04.496 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-04-02T11:32:04.496 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.496+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x17134d0 con 0x16741d0 2024-04-02T11:32:04.507 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.507+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 9 ==== osd_op_reply(10 exist [write 0~5] v237'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.507 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteFuture (11 ms) 2024-04-02T11:32:04.507 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-04-02T11:32:04.507 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.507+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1735bc0 con 0x16741d0 2024-04-02T11:32:04.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.519+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 10 ==== osd_op_reply(11 exist [write 0~5] v237'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.520 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteYield (12 ms) 2024-04-02T11:32:04.520 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-04-02T11:32:04.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.519+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x17124c0 con 0x16741d0 2024-04-02T11:32:04.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.519+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1712e20 con 0x16741d0 2024-04-02T11:32:04.527 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.527+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.527 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.527+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.528 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (8 ms) 2024-04-02T11:32:04.528 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-04-02T11:32:04.528 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.527+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1712970 con 0x16741d0 2024-04-02T11:32:04.528 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.527+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1736240 con 0x16741d0 2024-04-02T11:32:04.538 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.538+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 13 ==== osd_op_reply(14 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.540 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.539+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 14 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.540 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (12 ms) 2024-04-02T11:32:04.540 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-04-02T11:32:04.540 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.540+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x1712970 con 0x16741d0 2024-04-02T11:32:04.540 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.540+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e236) -- 0x17130e0 con 0x16741d0 2024-04-02T11:32:04.547 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.547+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.547 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.547+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v237'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.548 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncReadOperationYield (8 ms) 2024-04-02T11:32:04.548 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-04-02T11:32:04.548 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.547+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1712970 con 0x16741d0 2024-04-02T11:32:04.559 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.559+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 17 ==== osd_op_reply(18 exist [writefull 0~5] v237'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.559 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (12 ms) 2024-04-02T11:32:04.559 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-04-02T11:32:04.560 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.559+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1712970 con 0x16741d0 2024-04-02T11:32:04.569 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.569+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 18 ==== osd_op_reply(19 exist [writefull 0~5] v237'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.570 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (10 ms) 2024-04-02T11:32:04.570 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-04-02T11:32:04.570 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.570+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 --> v2:172.21.15.133:6803/4121831248 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236) -- 0x1712970 con 0x16741d0 2024-04-02T11:32:04.581 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6249f9b640 1 -- 172.21.15.133:0/3667714673 <== osd.1 v2:172.21.15.133:6803/4121831248 19 ==== osd_op_reply(20 exist [writefull 0~5] v237'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f623c00a430 con 0x16741d0 2024-04-02T11:32:04.581 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (12 ms) 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 msgr2=0x17118c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 0x17118c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 msgr2=0x7f62280396d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 0x7f62280396d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 msgr2=0x17043b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 shutdown_connections 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f62280371c0 0x7f62280396d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> v2:172.21.15.133:6803/4121831248 conn(0x16741d0 0x17118c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 --2- 172.21.15.133:0/3667714673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x15ddbc0 0x17043b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 >> 172.21.15.133:0/3667714673 conn(0x14dd890 msgr2=0x1679840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 shutdown_connections 2024-04-02T11:32:04.583 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.581+0000 7f6251a3e6c0 1 -- 172.21.15.133:0/3667714673 wait complete. 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 12 tests from AsioRados (135 ms total) 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 12 tests from 1 test suite ran. (2370 ms total) 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 12 tests. 2024-04-02T11:32:04.584 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:32:04.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:32:04.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_list 2024-04-02T11:32:04.613 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.613+0000 7f159f4a89c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:04.613 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.613+0000 7f159f4a89c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:04.613 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:04.613+0000 7f159f4a89c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:04.614 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 11 tests from 3 test suites. 2024-04-02T11:32:04.614 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:32:04.614 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 7 tests from LibRadosList 2024-04-02T11:32:06.184 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.ListObjects 2024-04-02T11:32:06.828 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.ListObjects (644 ms) 2024-04-02T11:32:06.829 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-04-02T11:32:07.027 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (199 ms) 2024-04-02T11:32:07.027 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-04-02T11:32:07.113 INFO:tasks.workunit.client.0.smithi133.stdout:myset foo1,foo2,foo3 2024-04-02T11:32:07.284 INFO:tasks.workunit.client.0.smithi133.stdout:foo1 2024-04-02T11:32:07.285 INFO:tasks.workunit.client.0.smithi133.stdout:foo2 2024-04-02T11:32:07.285 INFO:tasks.workunit.client.0.smithi133.stdout:foo3 2024-04-02T11:32:07.285 INFO:tasks.workunit.client.0.smithi133.stdout:myset foo1,foo4,foo5 2024-04-02T11:32:07.374 INFO:tasks.workunit.client.0.smithi133.stdout:foo4 2024-04-02T11:32:07.374 INFO:tasks.workunit.client.0.smithi133.stdout:foo5 2024-04-02T11:32:07.374 INFO:tasks.workunit.client.0.smithi133.stdout:foo1 2024-04-02T11:32:07.374 INFO:tasks.workunit.client.0.smithi133.stdout:myset foo6,foo7 2024-04-02T11:32:07.462 INFO:tasks.workunit.client.0.smithi133.stdout:foo7 2024-04-02T11:32:07.463 INFO:tasks.workunit.client.0.smithi133.stdout:foo6 2024-04-02T11:32:07.463 INFO:tasks.workunit.client.0.smithi133.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout:ns1:foo4 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout:ns1:foo5 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout:ns2:foo7 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout:ns2:foo6 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout:ns1:foo1 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout::foo1 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout::foo2 2024-04-02T11:32:07.552 INFO:tasks.workunit.client.0.smithi133.stdout::foo3 2024-04-02T11:32:07.721 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.ListObjectsNS (693 ms) 2024-04-02T11:32:07.721 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:1 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:10 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:13 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:7 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:14 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:0 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:15 0 2024-04-02T11:32:07.983 INFO:tasks.workunit.client.0.smithi133.stdout:11 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:5 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:8 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:6 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:3 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:4 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:12 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:9 0 2024-04-02T11:32:07.984 INFO:tasks.workunit.client.0.smithi133.stdout:2 0 2024-04-02T11:32:08.072 INFO:tasks.workunit.client.0.smithi133.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-02T11:32:08.313 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.ListObjectsStart (593 ms) 2024-04-02T11:32:08.313 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-04-02T11:32:08.626 INFO:tasks.workunit.client.0.smithi133.stdout:x cursor=MIN 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-04-02T11:32:08.717 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-04-02T11:32:08.718 INFO:tasks.workunit.client.0.smithi133.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-04-02T11:32:08.803 INFO:tasks.workunit.client.0.smithi133.stdout:FIRST> seek to MIN oid=1 2024-04-02T11:32:08.891 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-04-02T11:32:08.891 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:02547ec2:::1:head 2024-04-02T11:32:08.978 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:02547ec2:::1:head -> 1 2024-04-02T11:32:08.979 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-04-02T11:32:08.979 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:52ea6a34:::10:head 2024-04-02T11:32:09.038 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:52ea6a34:::10:head -> 10 2024-04-02T11:32:09.038 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-04-02T11:32:09.039 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:566253c9:::13:head 2024-04-02T11:32:09.099 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:566253c9:::13:head -> 13 2024-04-02T11:32:09.099 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-04-02T11:32:09.099 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:5c6b0b28:::7:head 2024-04-02T11:32:09.156 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:5c6b0b28:::7:head -> 7 2024-04-02T11:32:09.156 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-04-02T11:32:09.156 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:62a1935d:::14:head 2024-04-02T11:32:09.212 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:62a1935d:::14:head -> 14 2024-04-02T11:32:09.212 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-04-02T11:32:09.213 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:6cac518f:::0:head 2024-04-02T11:32:09.264 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:6cac518f:::0:head -> 0 2024-04-02T11:32:09.264 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-04-02T11:32:09.264 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:863748b0:::15:head 2024-04-02T11:32:09.307 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:863748b0:::15:head -> 15 2024-04-02T11:32:09.308 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-04-02T11:32:09.308 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:89d3ae78:::11:head 2024-04-02T11:32:09.347 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:89d3ae78:::11:head -> 11 2024-04-02T11:32:09.347 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-04-02T11:32:09.347 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:b29083e3:::5:head 2024-04-02T11:32:09.374 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:b29083e3:::5:head -> 5 2024-04-02T11:32:09.374 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-04-02T11:32:09.374 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:bd63b0f1:::8:head 2024-04-02T11:32:09.397 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:bd63b0f1:::8:head -> 8 2024-04-02T11:32:09.398 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-04-02T11:32:09.398 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:c4fdafeb:::6:head 2024-04-02T11:32:09.419 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:c4fdafeb:::6:head -> 6 2024-04-02T11:32:09.419 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-04-02T11:32:09.419 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:cfc208b3:::3:head 2024-04-02T11:32:09.439 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:cfc208b3:::3:head -> 3 2024-04-02T11:32:09.439 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-04-02T11:32:09.439 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:d83876eb:::4:head 2024-04-02T11:32:09.453 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:d83876eb:::4:head -> 4 2024-04-02T11:32:09.453 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-04-02T11:32:09.454 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:de5d7c5f:::12:head 2024-04-02T11:32:09.468 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:de5d7c5f:::12:head -> 12 2024-04-02T11:32:09.468 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-04-02T11:32:09.468 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:e960b815:::9:head 2024-04-02T11:32:09.476 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:e960b815:::9:head -> 9 2024-04-02T11:32:09.476 INFO:tasks.workunit.client.0.smithi133.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-04-02T11:32:09.477 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:f905c69b:::2:head 2024-04-02T11:32:09.480 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:f905c69b:::2:head -> 2 2024-04-02T11:32:09.480 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:52ea6a34:::10:head 2024-04-02T11:32:09.480 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-04-02T11:32:09.540 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:52ea6a34:::10:head -> 10 2024-04-02T11:32:09.540 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=10 expected=10 2024-04-02T11:32:09.540 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:02547ec2:::1:head 2024-04-02T11:32:09.540 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-04-02T11:32:09.630 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:02547ec2:::1:head -> 1 2024-04-02T11:32:09.630 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=1 expected=1 2024-04-02T11:32:09.630 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:5c6b0b28:::7:head 2024-04-02T11:32:09.630 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-04-02T11:32:09.687 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:5c6b0b28:::7:head -> 7 2024-04-02T11:32:09.687 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=7 expected=7 2024-04-02T11:32:09.687 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:566253c9:::13:head 2024-04-02T11:32:09.687 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-04-02T11:32:09.747 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:566253c9:::13:head -> 13 2024-04-02T11:32:09.747 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=13 expected=13 2024-04-02T11:32:09.747 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:e960b815:::9:head 2024-04-02T11:32:09.747 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-04-02T11:32:09.755 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:e960b815:::9:head -> 9 2024-04-02T11:32:09.756 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=9 expected=9 2024-04-02T11:32:09.756 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:c4fdafeb:::6:head 2024-04-02T11:32:09.756 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-04-02T11:32:09.778 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:c4fdafeb:::6:head -> 6 2024-04-02T11:32:09.778 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=6 expected=6 2024-04-02T11:32:09.778 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:f905c69b:::2:head 2024-04-02T11:32:09.778 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-04-02T11:32:09.781 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:f905c69b:::2:head -> 2 2024-04-02T11:32:09.781 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=2 expected=2 2024-04-02T11:32:09.781 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:bd63b0f1:::8:head 2024-04-02T11:32:09.781 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-04-02T11:32:09.805 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:bd63b0f1:::8:head -> 8 2024-04-02T11:32:09.806 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=8 expected=8 2024-04-02T11:32:09.806 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:de5d7c5f:::12:head 2024-04-02T11:32:09.806 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-04-02T11:32:09.820 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:de5d7c5f:::12:head -> 12 2024-04-02T11:32:09.820 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=12 expected=12 2024-04-02T11:32:09.820 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:d83876eb:::4:head 2024-04-02T11:32:09.820 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-04-02T11:32:09.834 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:d83876eb:::4:head -> 4 2024-04-02T11:32:09.834 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=4 expected=4 2024-04-02T11:32:09.835 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:cfc208b3:::3:head 2024-04-02T11:32:09.835 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-04-02T11:32:09.854 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:cfc208b3:::3:head -> 3 2024-04-02T11:32:09.854 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=3 expected=3 2024-04-02T11:32:09.854 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:b29083e3:::5:head 2024-04-02T11:32:09.854 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-04-02T11:32:09.881 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:b29083e3:::5:head -> 5 2024-04-02T11:32:09.881 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=5 expected=5 2024-04-02T11:32:09.881 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:89d3ae78:::11:head 2024-04-02T11:32:09.881 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-04-02T11:32:09.921 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:89d3ae78:::11:head -> 11 2024-04-02T11:32:09.921 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=11 expected=11 2024-04-02T11:32:09.921 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:863748b0:::15:head 2024-04-02T11:32:09.921 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-04-02T11:32:09.965 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:863748b0:::15:head -> 15 2024-04-02T11:32:09.965 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=15 expected=15 2024-04-02T11:32:09.965 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:6cac518f:::0:head 2024-04-02T11:32:09.965 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-04-02T11:32:10.016 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:6cac518f:::0:head -> 0 2024-04-02T11:32:10.016 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=0 expected=0 2024-04-02T11:32:10.016 INFO:tasks.workunit.client.0.smithi133.stdout:: seek to 76:62a1935d:::14:head 2024-04-02T11:32:10.016 INFO:tasks.workunit.client.0.smithi133.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-04-02T11:32:10.071 INFO:tasks.workunit.client.0.smithi133.stdout:> 76:62a1935d:::14:head -> 14 2024-04-02T11:32:10.071 INFO:tasks.workunit.client.0.smithi133.stdout:: entry=14 expected=14 2024-04-02T11:32:10.314 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.ListObjectsCursor (2000 ms) 2024-04-02T11:32:10.314 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-04-02T11:32:10.818 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.EnumerateObjects (505 ms) 2024-04-02T11:32:10.818 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-04-02T11:32:10.981 INFO:tasks.workunit.client.0.smithi133.stdout:split 0/5 -> MIN 76:33333333::::head 2024-04-02T11:32:11.002 INFO:tasks.workunit.client.0.smithi133.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-04-02T11:32:11.023 INFO:tasks.workunit.client.0.smithi133.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-04-02T11:32:11.047 INFO:tasks.workunit.client.0.smithi133.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-04-02T11:32:11.065 INFO:tasks.workunit.client.0.smithi133.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-04-02T11:32:11.342 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (524 ms) 2024-04-02T11:32:12.203 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 7 tests from LibRadosList (5158 ms total) 2024-04-02T11:32:12.203 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:12.203 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 3 tests from LibRadosListEC 2024-04-02T11:32:12.203 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:70: Skipped 2024-04-02T11:32:12.203 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:84: Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:96: Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:84: Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:96: Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-04-02T11:32:12.204 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:84: Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:96: Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:78: Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosListNP 2024-04-02T11:32:12.205 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosListNP.ListObjectsError (3024 ms) 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosListNP (3024 ms total) 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 11 tests from 3 test suites ran. (10613 ms total) 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 8 tests. 2024-04-02T11:32:15.226 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 3 tests, listed below: 2024-04-02T11:32:15.227 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-04-02T11:32:15.227 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-04-02T11:32:15.227 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-04-02T11:32:15.228 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:32:15.228 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:32:15.228 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_lock 2024-04-02T11:32:15.247 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:32:15.247 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 16 tests from 2 test suites. 2024-04-02T11:32:15.247 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:32:15.247 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLock 2024-04-02T11:32:17.235 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-04-02T11:32:17.578 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.LockExclusive (343 ms) 2024-04-02T11:32:17.578 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.LockShared 2024-04-02T11:32:17.625 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.LockShared (47 ms) 2024-04-02T11:32:17.625 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-04-02T11:32:18.658 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1033 ms) 2024-04-02T11:32:18.658 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-04-02T11:32:19.691 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.LockSharedDur (1033 ms) 2024-04-02T11:32:19.691 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-04-02T11:32:19.726 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.LockMayRenew (35 ms) 2024-04-02T11:32:19.726 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.Unlock 2024-04-02T11:32:19.764 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.Unlock (38 ms) 2024-04-02T11:32:19.764 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.ListLockers 2024-04-02T11:32:19.822 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.ListLockers (58 ms) 2024-04-02T11:32:19.822 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLock.BreakLock 2024-04-02T11:32:19.851 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLock.BreakLock (28 ms) 2024-04-02T11:32:20.247 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLock (2615 ms total) 2024-04-02T11:32:20.247 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:20.247 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockEC 2024-04-02T11:32:20.247 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:168: Skipped 2024-04-02T11:32:20.247 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-04-02T11:32:20.248 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.249 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-04-02T11:32:20.250 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:182: Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:196: Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/TestCase.cc:176: Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 16 tests from 2 test suites ran. (4999 ms total) 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 8 tests. 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 8 tests, listed below: 2024-04-02T11:32:20.251 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-04-02T11:32:20.252 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:32:20.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:32:20.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_lock_pp 2024-04-02T11:32:20.266 INFO:tasks.workunit.client.0.smithi133.stdout:Running main() from gmock_main.cc 2024-04-02T11:32:20.266 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 16 tests from 2 test suites. 2024-04-02T11:32:20.266 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:32:20.266 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockPP 2024-04-02T11:32:20.266 INFO:tasks.workunit.client.0.smithi133.stdout:seed 34273 2024-04-02T11:32:22.255 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-04-02T11:32:22.613 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (358 ms) 2024-04-02T11:32:22.614 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-04-02T11:32:22.636 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.LockSharedPP (23 ms) 2024-04-02T11:32:22.636 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-04-02T11:32:23.678 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1042 ms) 2024-04-02T11:32:23.679 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-04-02T11:32:24.712 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1034 ms) 2024-04-02T11:32:24.712 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-04-02T11:32:24.745 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (33 ms) 2024-04-02T11:32:24.745 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-04-02T11:32:24.781 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.UnlockPP (36 ms) 2024-04-02T11:32:24.781 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-04-02T11:32:24.823 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.ListLockersPP (42 ms) 2024-04-02T11:32:24.823 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-04-02T11:32:24.851 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosLockPP.BreakLockPP (28 ms) 2024-04-02T11:32:25.268 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockPP (2596 ms total) 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockECPP 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:373: Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-04-02T11:32:25.269 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.270 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-04-02T11:32:25.271 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:387: Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:400: Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/testcase_cxx.cc:381: Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosLockECPP (0 ms total) 2024-04-02T11:32:25.272 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 16 tests from 2 test suites ran. (5000 ms total) 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 8 tests. 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] 8 tests, listed below: 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-04-02T11:32:25.273 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-04-02T11:32:25.274 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:32:25.274 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:32:25.274 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_misc 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:25.300+0000 7f636987a6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:25.300+0000 7f636987a6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:25.301+0000 7f636987a6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 12 tests from 4 test suites. 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:32:25.301 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscVersion 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-04-02T11:32:25.302 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-04-02T11:32:25.343 INFO:tasks.workunit.client.0.smithi133.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-04-02T11:32:25.343 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:25.343+0000 7f636987a6c0 -1 failed for service _ceph-mon._tcp 2024-04-02T11:32:25.344 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:32:25.343+0000 7f636987a6c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-04-02T11:32:25.344 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (43 ms) 2024-04-02T11:32:25.344 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-04-02T11:32:27.355 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2011 ms) 2024-04-02T11:32:27.356 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2054 ms total) 2024-04-02T11:32:27.356 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:27.356 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscPool 2024-04-02T11:32:27.356 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-04-02T11:32:30.279 INFO:tasks.workunit.client.0.smithi133.stdout:started 0x7f6338005de0 2024-04-02T11:32:31.279 INFO:tasks.workunit.client.0.smithi133.stdout:started 0x7f6348005d20 2024-04-02T11:32:31.280 INFO:tasks.workunit.client.0.smithi133.stdout: started 2 aios 2024-04-02T11:32:31.280 INFO:tasks.workunit.client.0.smithi133.stdout:waiting 0x7f6338005de0 2024-04-02T11:32:31.280 INFO:tasks.workunit.client.0.smithi133.stdout:waiting 0x7f6348005d20 2024-04-02T11:32:31.283 INFO:tasks.workunit.client.0.smithi133.stdout:done. 2024-04-02T11:32:33.294 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (5939 ms) 2024-04-02T11:32:33.294 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscPool (5939 ms total) 2024-04-02T11:32:33.295 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:32:33.295 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosMisc 2024-04-02T11:32:35.034 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-04-02T11:32:35.034 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-04-02T11:32:35.034 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.Exec 2024-04-02T11:32:35.422 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.Exec (388 ms) 2024-04-02T11:32:35.422 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-04-02T11:32:35.462 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.WriteSame (41 ms) 2024-04-02T11:32:35.462 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-04-02T11:32:35.481 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.CmpExt (19 ms) 2024-04-02T11:32:35.482 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.Applications 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.Applications (4573 ms) 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-04-02T11:32:40.055 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-04-02T11:33:05.499 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMisc.ShutdownRace (25444 ms) 2024-04-02T11:33:06.078 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 8 tests from LibRadosMisc (30466 ms total) 2024-04-02T11:33:06.078 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:06.078 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment tear-down 2024-04-02T11:33:06.078 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] 12 tests from 4 test suites ran. (40777 ms total) 2024-04-02T11:33:06.078 INFO:tasks.workunit.client.0.smithi133.stdout:[ PASSED ] 12 tests. 2024-04-02T11:33:06.087 INFO:tasks.workunit.client.0.smithi133.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-02T11:33:06.088 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -eq 1 ']' 2024-04-02T11:33:06.088 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph_test_rados_api_misc_pp 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:33:06.118+0000 7f2f3c82a840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:33:06.118+0000 7f2f3c82a840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stderr:2024-04-02T11:33:06.119+0000 7f2f3c82a840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stdout:[==========] Running 31 tests from 7 test suites. 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] Global test environment set-up. 2024-04-02T11:33:06.119 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscVersion 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 22 tests from LibRadosMiscPP 2024-04-02T11:33:06.120 INFO:tasks.workunit.client.0.smithi133.stdout:seed 95939 2024-04-02T11:33:08.087 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-04-02T11:33:08.088 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-04-02T11:33:08.088 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-04-02T11:33:08.556 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.LongNamePP (469 ms) 2024-04-02T11:33:08.556 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-04-02T11:33:08.557 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:58: Failure 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout:Expected equality of these values: 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout: 0 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout: ioctx.write( string("a").c_str(), bl, bl.length(), 0) 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout: Which is: -95 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout:[ FAILED ] LibRadosMiscPP.LongLocatorPP (2 ms) 2024-04-02T11:33:08.558 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-04-02T11:33:08.628 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (70 ms) 2024-04-02T11:33:08.628 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-04-02T11:33:08.673 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (45 ms) 2024-04-02T11:33:08.673 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-04-02T11:33:08.689 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.ExecPP (16 ms) 2024-04-02T11:33:08.689 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-04-02T11:33:08.705 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (15 ms) 2024-04-02T11:33:08.705 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-04-02T11:33:08.732 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.Operate1PP (28 ms) 2024-04-02T11:33:08.732 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-04-02T11:33:08.748 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.Operate2PP (16 ms) 2024-04-02T11:33:08.748 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-04-02T11:33:08.794 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (46 ms) 2024-04-02T11:33:08.794 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-04-02T11:33:08.810 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (16 ms) 2024-04-02T11:33:08.810 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-04-02T11:33:08.833 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (23 ms) 2024-04-02T11:33:08.833 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-04-02T11:33:08.878 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (45 ms) 2024-04-02T11:33:08.878 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-04-02T11:33:08.889 INFO:tasks.workunit.client.0.smithi133.stdout:osd_max_attr_size = 0 2024-04-02T11:33:08.889 INFO:tasks.workunit.client.0.smithi133.stdout:osd_max_attr_size == 0; skipping test 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (18573 ms) 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:386: Skipped 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:549: Skipped 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-04-02T11:33:27.452 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-04-02T11:33:27.496 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (45 ms) 2024-04-02T11:33:27.497 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-04-02T11:33:27.520 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (24 ms) 2024-04-02T11:33:27.520 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-04-02T11:33:32.048 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.Applications (4528 ms) 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-04-02T11:33:32.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 22 tests from LibRadosMiscPP (23961 ms total) 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:476: Skipped 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-04-02T11:33:33.050 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:498: Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:504: Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2589-g38caedd8/rpm/el9/BUILD/ceph-19.0.0-2589-g38caedd8/src/test/librados/misc_cxx.cc:491: Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:Not supported by crimson yet. Skipped 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:33.051 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-04-02T11:33:35.063 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-04-02T11:33:35.403 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosChecksum/0.Subset (340 ms) 2024-04-02T11:33:35.403 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-04-02T11:33:35.429 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosChecksum/0.Chunked (26 ms) 2024-04-02T11:33:36.065 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosChecksum/0 (366 ms total) 2024-04-02T11:33:36.065 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:36.065 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-04-02T11:33:38.075 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-04-02T11:33:38.374 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosChecksum/1.Subset (299 ms) 2024-04-02T11:33:38.374 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-04-02T11:33:38.494 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosChecksum/1.Chunked (120 ms) 2024-04-02T11:33:39.045 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosChecksum/1 (419 ms total) 2024-04-02T11:33:39.045 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-04-02T11:33:39.045 INFO:tasks.workunit.client.0.smithi133.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-04-02T11:33:41.049 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-04-02T11:33:41.391 INFO:tasks.workunit.client.0.smithi133.stdout:[ OK ] LibRadosChecksum/2.Subset (342 ms) 2024-04-02T11:33:41.391 INFO:tasks.workunit.client.0.smithi133.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-04-02T14:27:59.583 INFO:tasks.workunit.client.0.smithi133.stderr:++ cleanup 2024-04-02T14:27:59.612 INFO:tasks.workunit.client.0.smithi133.stderr:++ pkill -P 32877 2024-04-02T14:27:59.613 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-02T14:27:59.613 INFO:tasks.workunit.client.0.smithi133.stderr:Terminated 2024-04-02T14:27:59.613 INFO:tasks.workunit.client.0.smithi133.stderr:++ true 2024-04-02T14:27:59.614 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-04-02T14:27:59.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T14:27:59.957 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi133 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=107e1624f2a9b72a56ee529969558944d24a2a75 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-04-02T14:28:00.184 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1e9da25bf98944319d992954582d67cf Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_107e1624f2a9b72a56ee529969558944d24a2a75/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi133 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=107e1624f2a9b72a56ee529969558944d24a2a75 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-04-02T14:28:00.186 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-02T14:28:00.198 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-02T14:28:00.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:28:00.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.309+0000 7f6fbc617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.314+0000 7f6fbc617640 1 Processor -- start 2024-04-02T14:28:00.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.314+0000 7f6fbc617640 1 -- start start 2024-04-02T14:28:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fbc617640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6fb41313d0 0x7f6fb4135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fbc617640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb4130a60 con 0x7f6fb406e260 2024-04-02T14:28:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fbc617640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb409c8b0 con 0x7f6fb41313d0 2024-04-02T14:28:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fb9bb2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6fb41313d0 0x7f6fb4135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fba3b3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6fb406e260 0x7f6fb4130cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36980/0 (socket says 172.21.15.133:36980) 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fba3b3640 1 -- 172.21.15.133:0/1918157265 learned_addr learned my addr 172.21.15.133:0/1918157265 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fb9bb2640 1 -- 172.21.15.133:0/1918157265 >> v1:172.21.15.133:6789/0 conn(0x7f6fb406e260 legacy=0x7f6fb4130cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fb9bb2640 1 -- 172.21.15.133:0/1918157265 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb409c2f0 con 0x7f6fb41313d0 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.315+0000 7f6fb93b1640 1 -- 172.21.15.133:0/1918157265 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127868501 0 0) 0x7f6fb4130a60 con 0x7f6fb406e260 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fb9bb2640 1 --2- 172.21.15.133:0/1918157265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb41313d0 0x7f6fb4135dc0 crc :-1 s=READY pgs=12308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e57cf2f948db84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fb93b1640 1 -- 172.21.15.133:0/1918157265 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6fa4009d90 con 0x7f6fb41313d0 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fb93b1640 1 -- 172.21.15.133:0/1918157265 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6fa400e230 con 0x7f6fb41313d0 2024-04-02T14:28:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fb93b1640 1 -- 172.21.15.133:0/1918157265 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6fa400e4d0 con 0x7f6fb41313d0 2024-04-02T14:28:00.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fbc617640 1 -- 172.21.15.133:0/1918157265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb41313d0 msgr2=0x7f6fb4135dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fbc617640 1 --2- 172.21.15.133:0/1918157265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb41313d0 0x7f6fb4135dc0 crc :-1 s=READY pgs=12308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fbc617640 1 -- 172.21.15.133:0/1918157265 shutdown_connections 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.316+0000 7f6fbc617640 1 --2- 172.21.15.133:0/1918157265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb41313d0 0x7f6fb4135dc0 unknown :-1 s=CLOSED pgs=12308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.317+0000 7f6fbc617640 1 -- 172.21.15.133:0/1918157265 >> 172.21.15.133:0/1918157265 conn(0x7f6fb40a0790 msgr2=0x7f6fb40a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.317+0000 7f6fbc617640 1 -- 172.21.15.133:0/1918157265 shutdown_connections 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.317+0000 7f6fbc617640 1 -- 172.21.15.133:0/1918157265 wait complete. 2024-04-02T14:28:00.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.317+0000 7f6fbc617640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.318+0000 7f6fbc617640 1 Processor -- start 2024-04-02T14:28:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.318+0000 7f6fbc617640 1 -- start start 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fbc617640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fbc617640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb4070a90 con 0x7f6fb406e260 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fba3b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fba3b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47702/0 (socket says 172.21.15.133:47702) 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fba3b3640 1 -- 172.21.15.133:0/3666692615 learned_addr learned my addr 172.21.15.133:0/3666692615 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.319+0000 7f6fba3b3640 1 -- 172.21.15.133:0/3666692615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb4072fe0 con 0x7f6fb406e260 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6fba3b3640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 crc :-1 s=READY pgs=12309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6fb000ec00 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6fb000ed60 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb406aa20 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f6fb0004840 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.320+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb4072130 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.321+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb4072e00 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.321+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f6fb00049f0 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.321+0000 7f6faaffd640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 0x7f6f9803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.321+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7f6fb004af00 con 0x7f6fb406e260 2024-04-02T14:28:00.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.321+0000 7f6fb9bb2640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 0x7f6f9803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.322+0000 7f6fb9bb2640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 0x7f6f9803ae90 crc :-1 s=READY pgs=3798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.326+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f6fb0001b50 con 0x7f6fb406e260 2024-04-02T14:28:00.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.501+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fb4136300 con 0x7f6f98038980 2024-04-02T14:28:00.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.507+0000 7f6faaffd640 1 -- 172.21.15.133:0/3666692615 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+187053 (crc 0 0 0) 0x7f6fa4009e80 con 0x7f6f98038980 2024-04-02T14:28:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:00.511 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T14:28:00.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 msgr2=0x7f6f9803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 0x7f6f9803ae90 crc :-1 s=READY pgs=3798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 msgr2=0x7f6fb406f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 crc :-1 s=READY pgs=12309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 shutdown_connections 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f6f98038980 0x7f6f9803ae90 unknown :-1 s=CLOSED pgs=3798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 --2- 172.21.15.133:0/3666692615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb406e260 0x7f6fb406f600 unknown :-1 s=CLOSED pgs=12309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 >> 172.21.15.133:0/3666692615 conn(0x7f6fb40a0790 msgr2=0x7f6fb412a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 shutdown_connections 2024-04-02T14:28:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.509+0000 7f6fbc617640 1 -- 172.21.15.133:0/3666692615 wait complete. 2024-04-02T14:28:00.522 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5713,"stamp":"2024-04-02T14:27:59.854995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594469,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":145,"num_write_kb":1905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":170848,"kb_used_data":6144,"kb_used_omap":0,"kb_used_meta":164672,"kb_avail":419259552,"statfs":{"total":429496729600,"available":429321781248,"internally_reserved":0,"allocated":6291456,"data_stored":2983642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":168624128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"86.2","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.060011+0000","last_change":"2024-04-02T11:33:41.348309+0000","last_active":"2024-04-02T11:33:44.060011+0000","last_peered":"2024-04-02T11:33:44.060011+0000","last_clean":"2024-04-02T11:33:44.060011+0000","last_became_active":"2024-04-02T11:33:41.342906+0000","last_became_peered":"2024-04-02T11:33:41.342906+0000","last_unstale":"2024-04-02T11:33:44.060011+0000","last_undegraded":"2024-04-02T11:33:44.060011+0000","last_fullsized":"2024-04-02T11:33:44.060011+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1f","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.083353+0000","last_change":"2024-04-02T11:32:04.380690+0000","last_active":"2024-04-02T11:33:44.083353+0000","last_peered":"2024-04-02T11:33:44.083353+0000","last_clean":"2024-04-02T11:33:44.083353+0000","last_became_active":"2024-04-02T11:32:04.372115+0000","last_became_peered":"2024-04-02T11:32:04.372115+0000","last_unstale":"2024-04-02T11:33:44.083353+0000","last_undegraded":"2024-04-02T11:33:44.083353+0000","last_fullsized":"2024-04-02T11:33:44.083353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.3","version":"285'1","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.111981+0000","last_change":"2024-04-02T11:33:41.302524+0000","last_active":"2024-04-02T11:33:44.111981+0000","last_peered":"2024-04-02T11:33:44.111981+0000","last_clean":"2024-04-02T11:33:44.111981+0000","last_became_active":"2024-04-02T11:33:41.286642+0000","last_became_peered":"2024-04-02T11:33:41.286642+0000","last_unstale":"2024-04-02T11:33:44.111981+0000","last_undegraded":"2024-04-02T11:33:44.111981+0000","last_fullsized":"2024-04-02T11:33:44.111981+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.479267+0000","last_change":"2024-04-02T11:32:04.439717+0000","last_active":"2024-04-02T11:33:44.479267+0000","last_peered":"2024-04-02T11:33:44.479267+0000","last_clean":"2024-04-02T11:33:44.479267+0000","last_became_active":"2024-04-02T11:32:04.424122+0000","last_became_peered":"2024-04-02T11:32:04.424122+0000","last_unstale":"2024-04-02T11:33:44.479267+0000","last_undegraded":"2024-04-02T11:33:44.479267+0000","last_fullsized":"2024-04-02T11:33:44.479267+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.0","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.109001+0000","last_change":"2024-04-02T11:33:41.305416+0000","last_active":"2024-04-02T11:33:44.109001+0000","last_peered":"2024-04-02T11:33:44.109001+0000","last_clean":"2024-04-02T11:33:44.109001+0000","last_became_active":"2024-04-02T11:33:41.268384+0000","last_became_peered":"2024-04-02T11:33:41.268384+0000","last_unstale":"2024-04-02T11:33:44.109001+0000","last_undegraded":"2024-04-02T11:33:44.109001+0000","last_fullsized":"2024-04-02T11:33:44.109001+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.608850+0000","last_change":"2024-04-02T11:32:04.410624+0000","last_active":"2024-04-02T11:33:44.608850+0000","last_peered":"2024-04-02T11:33:44.608850+0000","last_clean":"2024-04-02T11:33:44.608850+0000","last_became_active":"2024-04-02T11:32:04.394162+0000","last_became_peered":"2024-04-02T11:32:04.394162+0000","last_unstale":"2024-04-02T11:33:44.608850+0000","last_undegraded":"2024-04-02T11:33:44.608850+0000","last_fullsized":"2024-04-02T11:33:44.608850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.1","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773986+0000","last_change":"2024-04-02T11:33:41.330641+0000","last_active":"2024-04-02T11:33:44.773986+0000","last_peered":"2024-04-02T11:33:44.773986+0000","last_clean":"2024-04-02T11:33:44.773986+0000","last_became_active":"2024-04-02T11:33:41.317736+0000","last_became_peered":"2024-04-02T11:33:41.317736+0000","last_unstale":"2024-04-02T11:33:44.773986+0000","last_undegraded":"2024-04-02T11:33:44.773986+0000","last_fullsized":"2024-04-02T11:33:44.773986+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.771453+0000","last_change":"2024-04-02T11:32:04.455109+0000","last_active":"2024-04-02T11:33:44.771453+0000","last_peered":"2024-04-02T11:33:44.771453+0000","last_clean":"2024-04-02T11:33:44.771453+0000","last_became_active":"2024-04-02T11:32:04.430454+0000","last_became_peered":"2024-04-02T11:32:04.430454+0000","last_unstale":"2024-04-02T11:33:44.771453+0000","last_undegraded":"2024-04-02T11:33:44.771453+0000","last_fullsized":"2024-04-02T11:33:44.771453+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.6","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.574697+0000","last_change":"2024-04-02T11:33:41.252370+0000","last_active":"2024-04-02T11:33:44.574697+0000","last_peered":"2024-04-02T11:33:44.574697+0000","last_clean":"2024-04-02T11:33:44.574697+0000","last_became_active":"2024-04-02T11:33:41.236833+0000","last_became_peered":"2024-04-02T11:33:41.236833+0000","last_unstale":"2024-04-02T11:33:44.574697+0000","last_undegraded":"2024-04-02T11:33:44.574697+0000","last_fullsized":"2024-04-02T11:33:44.574697+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.115811+0000","last_change":"2024-04-02T11:32:04.397555+0000","last_active":"2024-04-02T11:33:44.115811+0000","last_peered":"2024-04-02T11:33:44.115811+0000","last_clean":"2024-04-02T11:33:44.115811+0000","last_became_active":"2024-04-02T11:32:04.392132+0000","last_became_peered":"2024-04-02T11:32:04.392132+0000","last_unstale":"2024-04-02T11:33:44.115811+0000","last_undegraded":"2024-04-02T11:33:44.115811+0000","last_fullsized":"2024-04-02T11:33:44.115811+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.7","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.708501+0000","last_change":"2024-04-02T11:33:41.301764+0000","last_active":"2024-04-02T11:33:44.708501+0000","last_peered":"2024-04-02T11:33:44.708501+0000","last_clean":"2024-04-02T11:33:44.708501+0000","last_became_active":"2024-04-02T11:33:41.288766+0000","last_became_peered":"2024-04-02T11:33:41.288766+0000","last_unstale":"2024-04-02T11:33:44.708501+0000","last_undegraded":"2024-04-02T11:33:44.708501+0000","last_fullsized":"2024-04-02T11:33:44.708501+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.464372+0000","last_change":"2024-04-02T11:32:04.448308+0000","last_active":"2024-04-02T11:33:44.464372+0000","last_peered":"2024-04-02T11:33:44.464372+0000","last_clean":"2024-04-02T11:33:44.464372+0000","last_became_active":"2024-04-02T11:32:04.435642+0000","last_became_peered":"2024-04-02T11:32:04.435642+0000","last_unstale":"2024-04-02T11:33:44.464372+0000","last_undegraded":"2024-04-02T11:33:44.464372+0000","last_fullsized":"2024-04-02T11:33:44.464372+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.4","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.746493+0000","last_change":"2024-04-02T11:33:41.237088+0000","last_active":"2024-04-02T11:33:44.746493+0000","last_peered":"2024-04-02T11:33:44.746493+0000","last_clean":"2024-04-02T11:33:44.746493+0000","last_became_active":"2024-04-02T11:33:41.230755+0000","last_became_peered":"2024-04-02T11:33:41.230755+0000","last_unstale":"2024-04-02T11:33:44.746493+0000","last_undegraded":"2024-04-02T11:33:44.746493+0000","last_fullsized":"2024-04-02T11:33:44.746493+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.476796+0000","last_change":"2024-04-02T11:32:04.531627+0000","last_active":"2024-04-02T11:33:44.476796+0000","last_peered":"2024-04-02T11:33:44.476796+0000","last_clean":"2024-04-02T11:33:44.476796+0000","last_became_active":"2024-04-02T11:32:04.523227+0000","last_became_peered":"2024-04-02T11:32:04.523227+0000","last_unstale":"2024-04-02T11:33:44.476796+0000","last_undegraded":"2024-04-02T11:33:44.476796+0000","last_fullsized":"2024-04-02T11:33:44.476796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.5","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.613522+0000","last_change":"2024-04-02T11:33:41.243595+0000","last_active":"2024-04-02T11:33:44.613522+0000","last_peered":"2024-04-02T11:33:44.613522+0000","last_clean":"2024-04-02T11:33:44.613522+0000","last_became_active":"2024-04-02T11:33:41.229522+0000","last_became_peered":"2024-04-02T11:33:41.229522+0000","last_unstale":"2024-04-02T11:33:44.613522+0000","last_undegraded":"2024-04-02T11:33:44.613522+0000","last_fullsized":"2024-04-02T11:33:44.613522+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.508048+0000","last_change":"2024-04-02T11:32:04.499203+0000","last_active":"2024-04-02T11:33:44.508048+0000","last_peered":"2024-04-02T11:33:44.508048+0000","last_clean":"2024-04-02T11:33:44.508048+0000","last_became_active":"2024-04-02T11:32:04.490360+0000","last_became_peered":"2024-04-02T11:32:04.490360+0000","last_unstale":"2024-04-02T11:33:44.508048+0000","last_undegraded":"2024-04-02T11:33:44.508048+0000","last_fullsized":"2024-04-02T11:33:44.508048+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.619749+0000","last_change":"2024-04-02T11:33:41.308637+0000","last_active":"2024-04-02T11:33:44.619749+0000","last_peered":"2024-04-02T11:33:44.619749+0000","last_clean":"2024-04-02T11:33:44.619749+0000","last_became_active":"2024-04-02T11:33:41.301597+0000","last_became_peered":"2024-04-02T11:33:41.301597+0000","last_unstale":"2024-04-02T11:33:44.619749+0000","last_undegraded":"2024-04-02T11:33:44.619749+0000","last_fullsized":"2024-04-02T11:33:44.619749+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.751562+0000","last_change":"2024-04-02T11:32:04.460524+0000","last_active":"2024-04-02T11:33:44.751562+0000","last_peered":"2024-04-02T11:33:44.751562+0000","last_clean":"2024-04-02T11:33:44.751562+0000","last_became_active":"2024-04-02T11:32:04.427391+0000","last_became_peered":"2024-04-02T11:32:04.427391+0000","last_unstale":"2024-04-02T11:33:44.751562+0000","last_undegraded":"2024-04-02T11:33:44.751562+0000","last_fullsized":"2024-04-02T11:33:44.751562+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.087235+0000","last_change":"2024-04-02T11:33:41.282206+0000","last_active":"2024-04-02T11:33:44.087235+0000","last_peered":"2024-04-02T11:33:44.087235+0000","last_clean":"2024-04-02T11:33:44.087235+0000","last_became_active":"2024-04-02T11:33:41.264133+0000","last_became_peered":"2024-04-02T11:33:41.264133+0000","last_unstale":"2024-04-02T11:33:44.087235+0000","last_undegraded":"2024-04-02T11:33:44.087235+0000","last_fullsized":"2024-04-02T11:33:44.087235+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.618701+0000","last_change":"2024-04-02T11:32:04.403716+0000","last_active":"2024-04-02T11:33:44.618701+0000","last_peered":"2024-04-02T11:33:44.618701+0000","last_clean":"2024-04-02T11:33:44.618701+0000","last_became_active":"2024-04-02T11:32:04.390779+0000","last_became_peered":"2024-04-02T11:32:04.390779+0000","last_unstale":"2024-04-02T11:33:44.618701+0000","last_undegraded":"2024-04-02T11:33:44.618701+0000","last_fullsized":"2024-04-02T11:33:44.618701+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.8","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.743680+0000","last_change":"2024-04-02T11:33:41.322754+0000","last_active":"2024-04-02T11:33:44.743680+0000","last_peered":"2024-04-02T11:33:44.743680+0000","last_clean":"2024-04-02T11:33:44.743680+0000","last_became_active":"2024-04-02T11:33:41.311852+0000","last_became_peered":"2024-04-02T11:33:41.311852+0000","last_unstale":"2024-04-02T11:33:44.743680+0000","last_undegraded":"2024-04-02T11:33:44.743680+0000","last_fullsized":"2024-04-02T11:33:44.743680+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.749745+0000","last_change":"2024-04-02T11:32:04.441727+0000","last_active":"2024-04-02T11:33:44.749745+0000","last_peered":"2024-04-02T11:33:44.749745+0000","last_clean":"2024-04-02T11:33:44.749745+0000","last_became_active":"2024-04-02T11:32:04.409646+0000","last_became_peered":"2024-04-02T11:32:04.409646+0000","last_unstale":"2024-04-02T11:33:44.749745+0000","last_undegraded":"2024-04-02T11:33:44.749745+0000","last_fullsized":"2024-04-02T11:33:44.749745+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.9","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773128+0000","last_change":"2024-04-02T11:33:41.293468+0000","last_active":"2024-04-02T11:33:44.773128+0000","last_peered":"2024-04-02T11:33:44.773128+0000","last_clean":"2024-04-02T11:33:44.773128+0000","last_became_active":"2024-04-02T11:33:41.267676+0000","last_became_peered":"2024-04-02T11:33:41.267676+0000","last_unstale":"2024-04-02T11:33:44.773128+0000","last_undegraded":"2024-04-02T11:33:44.773128+0000","last_fullsized":"2024-04-02T11:33:44.773128+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.509804+0000","last_change":"2024-04-02T11:32:04.404984+0000","last_active":"2024-04-02T11:33:44.509804+0000","last_peered":"2024-04-02T11:33:44.509804+0000","last_clean":"2024-04-02T11:33:44.509804+0000","last_became_active":"2024-04-02T11:32:04.387697+0000","last_became_peered":"2024-04-02T11:32:04.387697+0000","last_unstale":"2024-04-02T11:33:44.509804+0000","last_undegraded":"2024-04-02T11:33:44.509804+0000","last_fullsized":"2024-04-02T11:33:44.509804+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.755181+0000","last_change":"2024-04-02T11:33:41.296184+0000","last_active":"2024-04-02T11:33:44.755181+0000","last_peered":"2024-04-02T11:33:44.755181+0000","last_clean":"2024-04-02T11:33:44.755181+0000","last_became_active":"2024-04-02T11:33:41.271175+0000","last_became_peered":"2024-04-02T11:33:41.271175+0000","last_unstale":"2024-04-02T11:33:44.755181+0000","last_undegraded":"2024-04-02T11:33:44.755181+0000","last_fullsized":"2024-04-02T11:33:44.755181+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.732609+0000","last_change":"2024-04-02T11:32:04.423708+0000","last_active":"2024-04-02T11:33:44.732609+0000","last_peered":"2024-04-02T11:33:44.732609+0000","last_clean":"2024-04-02T11:33:44.732609+0000","last_became_active":"2024-04-02T11:32:04.401780+0000","last_became_peered":"2024-04-02T11:32:04.401780+0000","last_unstale":"2024-04-02T11:33:44.732609+0000","last_undegraded":"2024-04-02T11:33:44.732609+0000","last_fullsized":"2024-04-02T11:33:44.732609+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.742027+0000","last_change":"2024-04-02T11:33:41.274959+0000","last_active":"2024-04-02T11:33:44.742027+0000","last_peered":"2024-04-02T11:33:44.742027+0000","last_clean":"2024-04-02T11:33:44.742027+0000","last_became_active":"2024-04-02T11:33:41.254717+0000","last_became_peered":"2024-04-02T11:33:41.254717+0000","last_unstale":"2024-04-02T11:33:44.742027+0000","last_undegraded":"2024-04-02T11:33:44.742027+0000","last_fullsized":"2024-04-02T11:33:44.742027+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.728056+0000","last_change":"2024-04-02T11:32:04.457456+0000","last_active":"2024-04-02T11:33:44.728056+0000","last_peered":"2024-04-02T11:33:44.728056+0000","last_clean":"2024-04-02T11:33:44.728056+0000","last_became_active":"2024-04-02T11:32:04.436936+0000","last_became_peered":"2024-04-02T11:32:04.436936+0000","last_unstale":"2024-04-02T11:33:44.728056+0000","last_undegraded":"2024-04-02T11:33:44.728056+0000","last_fullsized":"2024-04-02T11:33:44.728056+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.579024+0000","last_change":"2024-04-02T11:33:41.259181+0000","last_active":"2024-04-02T11:33:44.579024+0000","last_peered":"2024-04-02T11:33:44.579024+0000","last_clean":"2024-04-02T11:33:44.579024+0000","last_became_active":"2024-04-02T11:33:41.249228+0000","last_became_peered":"2024-04-02T11:33:41.249228+0000","last_unstale":"2024-04-02T11:33:44.579024+0000","last_undegraded":"2024-04-02T11:33:44.579024+0000","last_fullsized":"2024-04-02T11:33:44.579024+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.503670+0000","last_change":"2024-04-02T11:32:04.429226+0000","last_active":"2024-04-02T11:33:44.503670+0000","last_peered":"2024-04-02T11:33:44.503670+0000","last_clean":"2024-04-02T11:33:44.503670+0000","last_became_active":"2024-04-02T11:32:04.407776+0000","last_became_peered":"2024-04-02T11:32:04.407776+0000","last_unstale":"2024-04-02T11:33:44.503670+0000","last_undegraded":"2024-04-02T11:33:44.503670+0000","last_fullsized":"2024-04-02T11:33:44.503670+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.480198+0000","last_change":"2024-04-02T11:33:41.329049+0000","last_active":"2024-04-02T11:33:44.480198+0000","last_peered":"2024-04-02T11:33:44.480198+0000","last_clean":"2024-04-02T11:33:44.480198+0000","last_became_active":"2024-04-02T11:33:41.323281+0000","last_became_peered":"2024-04-02T11:33:41.323281+0000","last_unstale":"2024-04-02T11:33:44.480198+0000","last_undegraded":"2024-04-02T11:33:44.480198+0000","last_fullsized":"2024-04-02T11:33:44.480198+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.494895+0000","last_change":"2024-04-02T11:32:04.558136+0000","last_active":"2024-04-02T11:33:44.494895+0000","last_peered":"2024-04-02T11:33:44.494895+0000","last_clean":"2024-04-02T11:33:44.494895+0000","last_became_active":"2024-04-02T11:32:04.551595+0000","last_became_peered":"2024-04-02T11:32:04.551595+0000","last_unstale":"2024-04-02T11:33:44.494895+0000","last_undegraded":"2024-04-02T11:33:44.494895+0000","last_fullsized":"2024-04-02T11:33:44.494895+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.12","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.478405+0000","last_change":"2024-04-02T11:33:41.281874+0000","last_active":"2024-04-02T11:33:44.478405+0000","last_peered":"2024-04-02T11:33:44.478405+0000","last_clean":"2024-04-02T11:33:44.478405+0000","last_became_active":"2024-04-02T11:33:41.274203+0000","last_became_peered":"2024-04-02T11:33:41.274203+0000","last_unstale":"2024-04-02T11:33:44.478405+0000","last_undegraded":"2024-04-02T11:33:44.478405+0000","last_fullsized":"2024-04-02T11:33:44.478405+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.485930+0000","last_change":"2024-04-02T11:32:04.417231+0000","last_active":"2024-04-02T11:33:44.485930+0000","last_peered":"2024-04-02T11:33:44.485930+0000","last_clean":"2024-04-02T11:33:44.485930+0000","last_became_active":"2024-04-02T11:32:04.396181+0000","last_became_peered":"2024-04-02T11:32:04.396181+0000","last_unstale":"2024-04-02T11:33:44.485930+0000","last_undegraded":"2024-04-02T11:33:44.485930+0000","last_fullsized":"2024-04-02T11:33:44.485930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.13","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.602064+0000","last_change":"2024-04-02T11:33:41.245713+0000","last_active":"2024-04-02T11:33:44.602064+0000","last_peered":"2024-04-02T11:33:44.602064+0000","last_clean":"2024-04-02T11:33:44.602064+0000","last_became_active":"2024-04-02T11:33:41.226115+0000","last_became_peered":"2024-04-02T11:33:41.226115+0000","last_unstale":"2024-04-02T11:33:44.602064+0000","last_undegraded":"2024-04-02T11:33:44.602064+0000","last_fullsized":"2024-04-02T11:33:44.602064+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.754146+0000","last_change":"2024-04-02T11:32:04.450092+0000","last_active":"2024-04-02T11:33:44.754146+0000","last_peered":"2024-04-02T11:33:44.754146+0000","last_clean":"2024-04-02T11:33:44.754146+0000","last_became_active":"2024-04-02T11:32:04.418023+0000","last_became_peered":"2024-04-02T11:32:04.418023+0000","last_unstale":"2024-04-02T11:33:44.754146+0000","last_undegraded":"2024-04-02T11:33:44.754146+0000","last_fullsized":"2024-04-02T11:33:44.754146+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.10","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.572668+0000","last_change":"2024-04-02T11:33:41.266477+0000","last_active":"2024-04-02T11:33:44.572668+0000","last_peered":"2024-04-02T11:33:44.572668+0000","last_clean":"2024-04-02T11:33:44.572668+0000","last_became_active":"2024-04-02T11:33:41.256091+0000","last_became_peered":"2024-04-02T11:33:41.256091+0000","last_unstale":"2024-04-02T11:33:44.572668+0000","last_undegraded":"2024-04-02T11:33:44.572668+0000","last_fullsized":"2024-04-02T11:33:44.572668+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.611376+0000","last_change":"2024-04-02T11:32:04.397771+0000","last_active":"2024-04-02T11:33:44.611376+0000","last_peered":"2024-04-02T11:33:44.611376+0000","last_clean":"2024-04-02T11:33:44.611376+0000","last_became_active":"2024-04-02T11:32:04.378859+0000","last_became_peered":"2024-04-02T11:32:04.378859+0000","last_unstale":"2024-04-02T11:33:44.611376+0000","last_undegraded":"2024-04-02T11:33:44.611376+0000","last_fullsized":"2024-04-02T11:33:44.611376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.11","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.758547+0000","last_change":"2024-04-02T11:33:41.263642+0000","last_active":"2024-04-02T11:33:44.758547+0000","last_peered":"2024-04-02T11:33:44.758547+0000","last_clean":"2024-04-02T11:33:44.758547+0000","last_became_active":"2024-04-02T11:33:41.250607+0000","last_became_peered":"2024-04-02T11:33:41.250607+0000","last_unstale":"2024-04-02T11:33:44.758547+0000","last_undegraded":"2024-04-02T11:33:44.758547+0000","last_fullsized":"2024-04-02T11:33:44.758547+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.073706+0000","last_change":"2024-04-02T11:32:04.463785+0000","last_active":"2024-04-02T11:33:44.073706+0000","last_peered":"2024-04-02T11:33:44.073706+0000","last_clean":"2024-04-02T11:33:44.073706+0000","last_became_active":"2024-04-02T11:32:04.457870+0000","last_became_peered":"2024-04-02T11:32:04.457870+0000","last_unstale":"2024-04-02T11:33:44.073706+0000","last_undegraded":"2024-04-02T11:33:44.073706+0000","last_fullsized":"2024-04-02T11:33:44.073706+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.16","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.605469+0000","last_change":"2024-04-02T11:33:41.241536+0000","last_active":"2024-04-02T11:33:44.605469+0000","last_peered":"2024-04-02T11:33:44.605469+0000","last_clean":"2024-04-02T11:33:44.605469+0000","last_became_active":"2024-04-02T11:33:41.232699+0000","last_became_peered":"2024-04-02T11:33:41.232699+0000","last_unstale":"2024-04-02T11:33:44.605469+0000","last_undegraded":"2024-04-02T11:33:44.605469+0000","last_fullsized":"2024-04-02T11:33:44.605469+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.071157+0000","last_change":"2024-04-02T11:32:04.384861+0000","last_active":"2024-04-02T11:33:44.071157+0000","last_peered":"2024-04-02T11:33:44.071157+0000","last_clean":"2024-04-02T11:33:44.071157+0000","last_became_active":"2024-04-02T11:32:04.368304+0000","last_became_peered":"2024-04-02T11:32:04.368304+0000","last_unstale":"2024-04-02T11:33:44.071157+0000","last_undegraded":"2024-04-02T11:33:44.071157+0000","last_fullsized":"2024-04-02T11:33:44.071157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.17","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.590092+0000","last_change":"2024-04-02T11:33:41.281940+0000","last_active":"2024-04-02T11:33:44.590092+0000","last_peered":"2024-04-02T11:33:44.590092+0000","last_clean":"2024-04-02T11:33:44.590092+0000","last_became_active":"2024-04-02T11:33:41.275074+0000","last_became_peered":"2024-04-02T11:33:41.275074+0000","last_unstale":"2024-04-02T11:33:44.590092+0000","last_undegraded":"2024-04-02T11:33:44.590092+0000","last_fullsized":"2024-04-02T11:33:44.590092+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.091094+0000","last_change":"2024-04-02T11:32:04.377416+0000","last_active":"2024-04-02T11:33:44.091094+0000","last_peered":"2024-04-02T11:33:44.091094+0000","last_clean":"2024-04-02T11:33:44.091094+0000","last_became_active":"2024-04-02T11:32:04.364464+0000","last_became_peered":"2024-04-02T11:32:04.364464+0000","last_unstale":"2024-04-02T11:33:44.091094+0000","last_undegraded":"2024-04-02T11:33:44.091094+0000","last_fullsized":"2024-04-02T11:33:44.091094+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.14","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.076833+0000","last_change":"2024-04-02T11:33:41.311816+0000","last_active":"2024-04-02T11:33:44.076833+0000","last_peered":"2024-04-02T11:33:44.076833+0000","last_clean":"2024-04-02T11:33:44.076833+0000","last_became_active":"2024-04-02T11:33:41.290381+0000","last_became_peered":"2024-04-02T11:33:41.290381+0000","last_unstale":"2024-04-02T11:33:44.076833+0000","last_undegraded":"2024-04-02T11:33:44.076833+0000","last_fullsized":"2024-04-02T11:33:44.076833+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.599260+0000","last_change":"2024-04-02T11:32:04.407271+0000","last_active":"2024-04-02T11:33:44.599260+0000","last_peered":"2024-04-02T11:33:44.599260+0000","last_clean":"2024-04-02T11:33:44.599260+0000","last_became_active":"2024-04-02T11:32:04.386636+0000","last_became_peered":"2024-04-02T11:32:04.386636+0000","last_unstale":"2024-04-02T11:33:44.599260+0000","last_undegraded":"2024-04-02T11:33:44.599260+0000","last_fullsized":"2024-04-02T11:33:44.599260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.15","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.085724+0000","last_change":"2024-04-02T11:33:41.299177+0000","last_active":"2024-04-02T11:33:44.085724+0000","last_peered":"2024-04-02T11:33:44.085724+0000","last_clean":"2024-04-02T11:33:44.085724+0000","last_became_active":"2024-04-02T11:33:41.271590+0000","last_became_peered":"2024-04-02T11:33:41.271590+0000","last_unstale":"2024-04-02T11:33:44.085724+0000","last_undegraded":"2024-04-02T11:33:44.085724+0000","last_fullsized":"2024-04-02T11:33:44.085724+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.703445+0000","last_change":"2024-04-02T11:32:04.421594+0000","last_active":"2024-04-02T11:33:44.703445+0000","last_peered":"2024-04-02T11:33:44.703445+0000","last_clean":"2024-04-02T11:33:44.703445+0000","last_became_active":"2024-04-02T11:32:04.404970+0000","last_became_peered":"2024-04-02T11:32:04.404970+0000","last_unstale":"2024-04-02T11:33:44.703445+0000","last_undegraded":"2024-04-02T11:33:44.703445+0000","last_fullsized":"2024-04-02T11:33:44.703445+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113166+0000","last_change":"2024-04-02T11:33:41.294167+0000","last_active":"2024-04-02T11:33:44.113166+0000","last_peered":"2024-04-02T11:33:44.113166+0000","last_clean":"2024-04-02T11:33:44.113166+0000","last_became_active":"2024-04-02T11:33:41.278749+0000","last_became_peered":"2024-04-02T11:33:41.278749+0000","last_unstale":"2024-04-02T11:33:44.113166+0000","last_undegraded":"2024-04-02T11:33:44.113166+0000","last_fullsized":"2024-04-02T11:33:44.113166+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.753274+0000","last_change":"2024-04-02T11:32:04.439591+0000","last_active":"2024-04-02T11:33:44.753274+0000","last_peered":"2024-04-02T11:33:44.753274+0000","last_clean":"2024-04-02T11:33:44.753274+0000","last_became_active":"2024-04-02T11:32:04.414684+0000","last_became_peered":"2024-04-02T11:32:04.414684+0000","last_unstale":"2024-04-02T11:33:44.753274+0000","last_undegraded":"2024-04-02T11:33:44.753274+0000","last_fullsized":"2024-04-02T11:33:44.753274+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.739455+0000","last_change":"2024-04-02T11:33:41.277360+0000","last_active":"2024-04-02T11:33:44.739455+0000","last_peered":"2024-04-02T11:33:44.739455+0000","last_clean":"2024-04-02T11:33:44.739455+0000","last_became_active":"2024-04-02T11:33:41.258789+0000","last_became_peered":"2024-04-02T11:33:41.258789+0000","last_unstale":"2024-04-02T11:33:44.739455+0000","last_undegraded":"2024-04-02T11:33:44.739455+0000","last_fullsized":"2024-04-02T11:33:44.739455+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.510847+0000","last_change":"2024-04-02T11:32:04.419077+0000","last_active":"2024-04-02T11:33:44.510847+0000","last_peered":"2024-04-02T11:33:44.510847+0000","last_clean":"2024-04-02T11:33:44.510847+0000","last_became_active":"2024-04-02T11:32:04.392023+0000","last_became_peered":"2024-04-02T11:32:04.392023+0000","last_unstale":"2024-04-02T11:33:44.510847+0000","last_undegraded":"2024-04-02T11:33:44.510847+0000","last_fullsized":"2024-04-02T11:33:44.510847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.18","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.501017+0000","last_change":"2024-04-02T11:33:41.379485+0000","last_active":"2024-04-02T11:33:44.501017+0000","last_peered":"2024-04-02T11:33:44.501017+0000","last_clean":"2024-04-02T11:33:44.501017+0000","last_became_active":"2024-04-02T11:33:41.374309+0000","last_became_peered":"2024-04-02T11:33:41.374309+0000","last_unstale":"2024-04-02T11:33:44.501017+0000","last_undegraded":"2024-04-02T11:33:44.501017+0000","last_fullsized":"2024-04-02T11:33:44.501017+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.766807+0000","last_change":"2024-04-02T11:32:04.464793+0000","last_active":"2024-04-02T11:33:44.766807+0000","last_peered":"2024-04-02T11:33:44.766807+0000","last_clean":"2024-04-02T11:33:44.766807+0000","last_became_active":"2024-04-02T11:32:04.445170+0000","last_became_peered":"2024-04-02T11:32:04.445170+0000","last_unstale":"2024-04-02T11:33:44.766807+0000","last_undegraded":"2024-04-02T11:33:44.766807+0000","last_fullsized":"2024-04-02T11:33:44.766807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.19","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.493963+0000","last_change":"2024-04-02T11:33:41.353471+0000","last_active":"2024-04-02T11:33:44.493963+0000","last_peered":"2024-04-02T11:33:44.493963+0000","last_clean":"2024-04-02T11:33:44.493963+0000","last_became_active":"2024-04-02T11:33:41.347559+0000","last_became_peered":"2024-04-02T11:33:41.347559+0000","last_unstale":"2024-04-02T11:33:44.493963+0000","last_undegraded":"2024-04-02T11:33:44.493963+0000","last_fullsized":"2024-04-02T11:33:44.493963+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.492318+0000","last_change":"2024-04-02T11:32:04.474632+0000","last_active":"2024-04-02T11:33:44.492318+0000","last_peered":"2024-04-02T11:33:44.492318+0000","last_clean":"2024-04-02T11:33:44.492318+0000","last_became_active":"2024-04-02T11:32:04.467593+0000","last_became_peered":"2024-04-02T11:32:04.467593+0000","last_unstale":"2024-04-02T11:33:44.492318+0000","last_undegraded":"2024-04-02T11:33:44.492318+0000","last_fullsized":"2024-04-02T11:33:44.492318+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.765959+0000","last_change":"2024-04-02T11:33:41.303867+0000","last_active":"2024-04-02T11:33:44.765959+0000","last_peered":"2024-04-02T11:33:44.765959+0000","last_clean":"2024-04-02T11:33:44.765959+0000","last_became_active":"2024-04-02T11:33:41.282720+0000","last_became_peered":"2024-04-02T11:33:41.282720+0000","last_unstale":"2024-04-02T11:33:44.765959+0000","last_undegraded":"2024-04-02T11:33:44.765959+0000","last_fullsized":"2024-04-02T11:33:44.765959+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.489009+0000","last_change":"2024-04-02T11:32:04.415345+0000","last_active":"2024-04-02T11:33:44.489009+0000","last_peered":"2024-04-02T11:33:44.489009+0000","last_clean":"2024-04-02T11:33:44.489009+0000","last_became_active":"2024-04-02T11:32:04.400375+0000","last_became_peered":"2024-04-02T11:32:04.400375+0000","last_unstale":"2024-04-02T11:33:44.489009+0000","last_undegraded":"2024-04-02T11:33:44.489009+0000","last_fullsized":"2024-04-02T11:33:44.489009+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.777113+0000","last_change":"2024-04-02T11:33:41.334657+0000","last_active":"2024-04-02T11:33:44.777113+0000","last_peered":"2024-04-02T11:33:44.777113+0000","last_clean":"2024-04-02T11:33:44.777113+0000","last_became_active":"2024-04-02T11:33:41.325633+0000","last_became_peered":"2024-04-02T11:33:41.325633+0000","last_unstale":"2024-04-02T11:33:44.777113+0000","last_undegraded":"2024-04-02T11:33:44.777113+0000","last_fullsized":"2024-04-02T11:33:44.777113+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.471747+0000","last_change":"2024-04-02T11:32:04.461967+0000","last_active":"2024-04-02T11:33:44.471747+0000","last_peered":"2024-04-02T11:33:44.471747+0000","last_clean":"2024-04-02T11:33:44.471747+0000","last_became_active":"2024-04-02T11:32:04.456013+0000","last_became_peered":"2024-04-02T11:32:04.456013+0000","last_unstale":"2024-04-02T11:33:44.471747+0000","last_undegraded":"2024-04-02T11:33:44.471747+0000","last_fullsized":"2024-04-02T11:33:44.471747+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.506496+0000","last_change":"2024-04-02T11:33:41.285447+0000","last_active":"2024-04-02T11:33:44.506496+0000","last_peered":"2024-04-02T11:33:44.506496+0000","last_clean":"2024-04-02T11:33:44.506496+0000","last_became_active":"2024-04-02T11:33:41.270749+0000","last_became_peered":"2024-04-02T11:33:41.270749+0000","last_unstale":"2024-04-02T11:33:44.506496+0000","last_undegraded":"2024-04-02T11:33:44.506496+0000","last_fullsized":"2024-04-02T11:33:44.506496+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":151,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.074943+0000","last_change":"2024-04-02T11:32:04.431094+0000","last_active":"2024-04-02T11:33:44.074943+0000","last_peered":"2024-04-02T11:33:44.074943+0000","last_clean":"2024-04-02T11:33:44.074943+0000","last_became_active":"2024-04-02T11:32:04.423058+0000","last_became_peered":"2024-04-02T11:32:04.423058+0000","last_unstale":"2024-04-02T11:33:44.074943+0000","last_undegraded":"2024-04-02T11:33:44.074943+0000","last_fullsized":"2024-04-02T11:33:44.074943+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.1d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.106087+0000","last_change":"2024-04-02T11:33:41.274565+0000","last_active":"2024-04-02T11:33:44.106087+0000","last_peered":"2024-04-02T11:33:44.106087+0000","last_clean":"2024-04-02T11:33:44.106087+0000","last_became_active":"2024-04-02T11:33:41.258583+0000","last_became_peered":"2024-04-02T11:33:41.258583+0000","last_unstale":"2024-04-02T11:33:44.106087+0000","last_undegraded":"2024-04-02T11:33:44.106087+0000","last_fullsized":"2024-04-02T11:33:44.106087+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.593632+0000","last_change":"2024-04-02T11:32:04.401482+0000","last_active":"2024-04-02T11:33:44.593632+0000","last_peered":"2024-04-02T11:33:44.593632+0000","last_clean":"2024-04-02T11:33:44.593632+0000","last_became_active":"2024-04-02T11:32:04.383344+0000","last_became_peered":"2024-04-02T11:32:04.383344+0000","last_unstale":"2024-04-02T11:33:44.593632+0000","last_undegraded":"2024-04-02T11:33:44.593632+0000","last_fullsized":"2024-04-02T11:33:44.593632+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.069239+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:33:44.069239+0000","last_peered":"2024-04-02T11:33:44.069239+0000","last_clean":"2024-04-02T11:33:44.069239+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:33:44.069239+0000","last_undegraded":"2024-04-02T11:33:44.069239+0000","last_fullsized":"2024-04-02T11:33:44.069239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.627702+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:33:44.627702+0000","last_peered":"2024-04-02T11:33:44.627702+0000","last_clean":"2024-04-02T11:33:44.627702+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:33:44.627702+0000","last_undegraded":"2024-04-02T11:33:44.627702+0000","last_fullsized":"2024-04-02T11:33:44.627702+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":741,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.722011+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:33:44.722011+0000","last_peered":"2024-04-02T11:33:44.722011+0000","last_clean":"2024-04-02T11:33:44.722011+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:33:44.722011+0000","last_undegraded":"2024-04-02T11:33:44.722011+0000","last_fullsized":"2024-04-02T11:33:44.722011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.623600+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:33:44.623600+0000","last_peered":"2024-04-02T11:33:44.623600+0000","last_clean":"2024-04-02T11:33:44.623600+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:33:44.623600+0000","last_undegraded":"2024-04-02T11:33:44.623600+0000","last_fullsized":"2024-04-02T11:33:44.623600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.473736+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:33:44.473736+0000","last_peered":"2024-04-02T11:33:44.473736+0000","last_clean":"2024-04-02T11:33:44.473736+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:33:44.473736+0000","last_undegraded":"2024-04-02T11:33:44.473736+0000","last_fullsized":"2024-04-02T11:33:44.473736+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.110855+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:33:44.110855+0000","last_peered":"2024-04-02T11:33:44.110855+0000","last_clean":"2024-04-02T11:33:44.110855+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:33:44.110855+0000","last_undegraded":"2024-04-02T11:33:44.110855+0000","last_fullsized":"2024-04-02T11:33:44.110855+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.504908+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:33:44.504908+0000","last_peered":"2024-04-02T11:33:44.504908+0000","last_clean":"2024-04-02T11:33:44.504908+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:33:44.504908+0000","last_undegraded":"2024-04-02T11:33:44.504908+0000","last_fullsized":"2024-04-02T11:33:44.504908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113298+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:33:44.113298+0000","last_peered":"2024-04-02T11:33:44.113298+0000","last_clean":"2024-04-02T11:33:44.113298+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:33:44.113298+0000","last_undegraded":"2024-04-02T11:33:44.113298+0000","last_fullsized":"2024-04-02T11:33:44.113298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":877,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.734120+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:33:44.734120+0000","last_peered":"2024-04-02T11:33:44.734120+0000","last_clean":"2024-04-02T11:33:44.734120+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:33:44.734120+0000","last_undegraded":"2024-04-02T11:33:44.734120+0000","last_fullsized":"2024-04-02T11:33:44.734120+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":86,"num_pg":32,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":4},{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":31104,"kb_avail":104824672,"statfs":{"total":107374182400,"available":107340464128,"internally_reserved":0,"allocated":1859584,"data_stored":1037844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31850496},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829396,"statfs":{"total":107374182400,"available":107345301504,"internally_reserved":0,"allocated":1282048,"data_stored":451769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":54576,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104803024,"statfs":{"total":107374182400,"available":107318296576,"internally_reserved":0,"allocated":1286144,"data_stored":456057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":55140,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104802460,"statfs":{"total":107374182400,"available":107317719040,"internally_reserved":0,"allocated":1863680,"data_stored":1037972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:28:00.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:28:00.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.630+0000 7fe96935d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.635+0000 7fe96935d640 1 Processor -- start 2024-04-02T14:28:00.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.635+0000 7fe96935d640 1 -- start start 2024-04-02T14:28:00.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe96935d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe96409a9e0 0x7fe96409adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe96935d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe96409bdf0 con 0x7fe96409b330 2024-04-02T14:28:00.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe96935d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fe96409c020 con 0x7fe96409a9e0 2024-04-02T14:28:00.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe962ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe96409a9e0 0x7fe96409adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe9627fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe96409b330 0x7fe96412e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36992/0 (socket says 172.21.15.133:36992) 2024-04-02T14:28:00.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.636+0000 7fe9627fc640 1 -- 172.21.15.133:0/2041759079 learned_addr learned my addr 172.21.15.133:0/2041759079 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:00.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe962ffd640 1 -- 172.21.15.133:0/2041759079 >> v1:172.21.15.133:6789/0 conn(0x7fe96409b330 legacy=0x7fe96412e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe962ffd640 1 -- 172.21.15.133:0/2041759079 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96409d670 con 0x7fe96409a9e0 2024-04-02T14:28:00.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe962ffd640 1 --2- 172.21.15.133:0/2041759079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96409adf0 crc :-1 s=READY pgs=12311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add36c9eefb58274 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe961ffb640 1 -- 172.21.15.133:0/2041759079 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe954009d80 con 0x7fe96409a9e0 2024-04-02T14:28:00.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe961ffb640 1 -- 172.21.15.133:0/2041759079 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe95400e230 con 0x7fe96409a9e0 2024-04-02T14:28:00.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.637+0000 7fe961ffb640 1 -- 172.21.15.133:0/2041759079 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe95400e4d0 con 0x7fe96409a9e0 2024-04-02T14:28:00.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 -- 172.21.15.133:0/2041759079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 msgr2=0x7fe96409adf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 --2- 172.21.15.133:0/2041759079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96409adf0 crc :-1 s=READY pgs=12311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 -- 172.21.15.133:0/2041759079 shutdown_connections 2024-04-02T14:28:00.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 --2- 172.21.15.133:0/2041759079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96409adf0 unknown :-1 s=CLOSED pgs=12311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 -- 172.21.15.133:0/2041759079 >> 172.21.15.133:0/2041759079 conn(0x7fe9640a18a0 msgr2=0x7fe9640a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:00.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 -- 172.21.15.133:0/2041759079 shutdown_connections 2024-04-02T14:28:00.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 1 -- 172.21.15.133:0/2041759079 wait complete. 2024-04-02T14:28:00.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.638+0000 7fe96935d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.639+0000 7fe96935d640 1 Processor -- start 2024-04-02T14:28:00.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.640+0000 7fe96935d640 1 -- start start 2024-04-02T14:28:00.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.640+0000 7fe96935d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.640+0000 7fe96935d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe96406d390 con 0x7fe96409a9e0 2024-04-02T14:28:00.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe962ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe962ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47718/0 (socket says 172.21.15.133:47718) 2024-04-02T14:28:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe962ffd640 1 -- 172.21.15.133:0/2574397749 learned_addr learned my addr 172.21.15.133:0/2574397749 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe962ffd640 1 -- 172.21.15.133:0/2574397749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96406dfc0 con 0x7fe96409a9e0 2024-04-02T14:28:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe962ffd640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 crc :-1 s=READY pgs=12312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe954014620 con 0x7fe96409a9e0 2024-04-02T14:28:00.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe954014780 con 0x7fe96409a9e0 2024-04-02T14:28:00.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96411fdf0 con 0x7fe96409a9e0 2024-04-02T14:28:00.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe954014a20 con 0x7fe96409a9e0 2024-04-02T14:28:00.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.641+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96411f1e0 con 0x7fe96409a9e0 2024-04-02T14:28:00.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.642+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96411e200 con 0x7fe96409a9e0 2024-04-02T14:28:00.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.642+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fe954014bd0 con 0x7fe96409a9e0 2024-04-02T14:28:00.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.643+0000 7fe94b7fe640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 0x7fe95003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.643+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fe954019040 con 0x7fe96409a9e0 2024-04-02T14:28:00.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.643+0000 7fe9627fc640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 0x7fe95003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.643+0000 7fe9627fc640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 0x7fe95003ae90 crc :-1 s=READY pgs=3799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.648+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fe954012390 con 0x7fe96409a9e0 2024-04-02T14:28:00.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.820+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe96409adf0 con 0x7fe950038980 2024-04-02T14:28:00.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.826+0000 7fe94b7fe640 1 -- 172.21.15.133:0/2574397749 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+187053 (crc 0 0 0) 0x7fe96409adf0 con 0x7fe950038980 2024-04-02T14:28:00.831 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T14:28:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 msgr2=0x7fe95003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 0x7fe95003ae90 crc :-1 s=READY pgs=3799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 msgr2=0x7fe96411ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 crc :-1 s=READY pgs=12312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 shutdown_connections 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe950038980 0x7fe95003ae90 unknown :-1 s=CLOSED pgs=3799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 --2- 172.21.15.133:0/2574397749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96409a9e0 0x7fe96411ce40 unknown :-1 s=CLOSED pgs=12312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 >> 172.21.15.133:0/2574397749 conn(0x7fe9640a18a0 msgr2=0x7fe9641b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.828+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 shutdown_connections 2024-04-02T14:28:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.829+0000 7fe96935d640 1 -- 172.21.15.133:0/2574397749 wait complete. 2024-04-02T14:28:00.841 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5713,"stamp":"2024-04-02T14:27:59.854995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594469,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":145,"num_write_kb":1905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":170848,"kb_used_data":6144,"kb_used_omap":0,"kb_used_meta":164672,"kb_avail":419259552,"statfs":{"total":429496729600,"available":429321781248,"internally_reserved":0,"allocated":6291456,"data_stored":2983642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":168624128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"86.2","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.060011+0000","last_change":"2024-04-02T11:33:41.348309+0000","last_active":"2024-04-02T11:33:44.060011+0000","last_peered":"2024-04-02T11:33:44.060011+0000","last_clean":"2024-04-02T11:33:44.060011+0000","last_became_active":"2024-04-02T11:33:41.342906+0000","last_became_peered":"2024-04-02T11:33:41.342906+0000","last_unstale":"2024-04-02T11:33:44.060011+0000","last_undegraded":"2024-04-02T11:33:44.060011+0000","last_fullsized":"2024-04-02T11:33:44.060011+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1f","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.083353+0000","last_change":"2024-04-02T11:32:04.380690+0000","last_active":"2024-04-02T11:33:44.083353+0000","last_peered":"2024-04-02T11:33:44.083353+0000","last_clean":"2024-04-02T11:33:44.083353+0000","last_became_active":"2024-04-02T11:32:04.372115+0000","last_became_peered":"2024-04-02T11:32:04.372115+0000","last_unstale":"2024-04-02T11:33:44.083353+0000","last_undegraded":"2024-04-02T11:33:44.083353+0000","last_fullsized":"2024-04-02T11:33:44.083353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.3","version":"285'1","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.111981+0000","last_change":"2024-04-02T11:33:41.302524+0000","last_active":"2024-04-02T11:33:44.111981+0000","last_peered":"2024-04-02T11:33:44.111981+0000","last_clean":"2024-04-02T11:33:44.111981+0000","last_became_active":"2024-04-02T11:33:41.286642+0000","last_became_peered":"2024-04-02T11:33:41.286642+0000","last_unstale":"2024-04-02T11:33:44.111981+0000","last_undegraded":"2024-04-02T11:33:44.111981+0000","last_fullsized":"2024-04-02T11:33:44.111981+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.479267+0000","last_change":"2024-04-02T11:32:04.439717+0000","last_active":"2024-04-02T11:33:44.479267+0000","last_peered":"2024-04-02T11:33:44.479267+0000","last_clean":"2024-04-02T11:33:44.479267+0000","last_became_active":"2024-04-02T11:32:04.424122+0000","last_became_peered":"2024-04-02T11:32:04.424122+0000","last_unstale":"2024-04-02T11:33:44.479267+0000","last_undegraded":"2024-04-02T11:33:44.479267+0000","last_fullsized":"2024-04-02T11:33:44.479267+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.0","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.109001+0000","last_change":"2024-04-02T11:33:41.305416+0000","last_active":"2024-04-02T11:33:44.109001+0000","last_peered":"2024-04-02T11:33:44.109001+0000","last_clean":"2024-04-02T11:33:44.109001+0000","last_became_active":"2024-04-02T11:33:41.268384+0000","last_became_peered":"2024-04-02T11:33:41.268384+0000","last_unstale":"2024-04-02T11:33:44.109001+0000","last_undegraded":"2024-04-02T11:33:44.109001+0000","last_fullsized":"2024-04-02T11:33:44.109001+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.608850+0000","last_change":"2024-04-02T11:32:04.410624+0000","last_active":"2024-04-02T11:33:44.608850+0000","last_peered":"2024-04-02T11:33:44.608850+0000","last_clean":"2024-04-02T11:33:44.608850+0000","last_became_active":"2024-04-02T11:32:04.394162+0000","last_became_peered":"2024-04-02T11:32:04.394162+0000","last_unstale":"2024-04-02T11:33:44.608850+0000","last_undegraded":"2024-04-02T11:33:44.608850+0000","last_fullsized":"2024-04-02T11:33:44.608850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.1","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773986+0000","last_change":"2024-04-02T11:33:41.330641+0000","last_active":"2024-04-02T11:33:44.773986+0000","last_peered":"2024-04-02T11:33:44.773986+0000","last_clean":"2024-04-02T11:33:44.773986+0000","last_became_active":"2024-04-02T11:33:41.317736+0000","last_became_peered":"2024-04-02T11:33:41.317736+0000","last_unstale":"2024-04-02T11:33:44.773986+0000","last_undegraded":"2024-04-02T11:33:44.773986+0000","last_fullsized":"2024-04-02T11:33:44.773986+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.771453+0000","last_change":"2024-04-02T11:32:04.455109+0000","last_active":"2024-04-02T11:33:44.771453+0000","last_peered":"2024-04-02T11:33:44.771453+0000","last_clean":"2024-04-02T11:33:44.771453+0000","last_became_active":"2024-04-02T11:32:04.430454+0000","last_became_peered":"2024-04-02T11:32:04.430454+0000","last_unstale":"2024-04-02T11:33:44.771453+0000","last_undegraded":"2024-04-02T11:33:44.771453+0000","last_fullsized":"2024-04-02T11:33:44.771453+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.6","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.574697+0000","last_change":"2024-04-02T11:33:41.252370+0000","last_active":"2024-04-02T11:33:44.574697+0000","last_peered":"2024-04-02T11:33:44.574697+0000","last_clean":"2024-04-02T11:33:44.574697+0000","last_became_active":"2024-04-02T11:33:41.236833+0000","last_became_peered":"2024-04-02T11:33:41.236833+0000","last_unstale":"2024-04-02T11:33:44.574697+0000","last_undegraded":"2024-04-02T11:33:44.574697+0000","last_fullsized":"2024-04-02T11:33:44.574697+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.115811+0000","last_change":"2024-04-02T11:32:04.397555+0000","last_active":"2024-04-02T11:33:44.115811+0000","last_peered":"2024-04-02T11:33:44.115811+0000","last_clean":"2024-04-02T11:33:44.115811+0000","last_became_active":"2024-04-02T11:32:04.392132+0000","last_became_peered":"2024-04-02T11:32:04.392132+0000","last_unstale":"2024-04-02T11:33:44.115811+0000","last_undegraded":"2024-04-02T11:33:44.115811+0000","last_fullsized":"2024-04-02T11:33:44.115811+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.7","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.708501+0000","last_change":"2024-04-02T11:33:41.301764+0000","last_active":"2024-04-02T11:33:44.708501+0000","last_peered":"2024-04-02T11:33:44.708501+0000","last_clean":"2024-04-02T11:33:44.708501+0000","last_became_active":"2024-04-02T11:33:41.288766+0000","last_became_peered":"2024-04-02T11:33:41.288766+0000","last_unstale":"2024-04-02T11:33:44.708501+0000","last_undegraded":"2024-04-02T11:33:44.708501+0000","last_fullsized":"2024-04-02T11:33:44.708501+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.464372+0000","last_change":"2024-04-02T11:32:04.448308+0000","last_active":"2024-04-02T11:33:44.464372+0000","last_peered":"2024-04-02T11:33:44.464372+0000","last_clean":"2024-04-02T11:33:44.464372+0000","last_became_active":"2024-04-02T11:32:04.435642+0000","last_became_peered":"2024-04-02T11:32:04.435642+0000","last_unstale":"2024-04-02T11:33:44.464372+0000","last_undegraded":"2024-04-02T11:33:44.464372+0000","last_fullsized":"2024-04-02T11:33:44.464372+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.4","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.746493+0000","last_change":"2024-04-02T11:33:41.237088+0000","last_active":"2024-04-02T11:33:44.746493+0000","last_peered":"2024-04-02T11:33:44.746493+0000","last_clean":"2024-04-02T11:33:44.746493+0000","last_became_active":"2024-04-02T11:33:41.230755+0000","last_became_peered":"2024-04-02T11:33:41.230755+0000","last_unstale":"2024-04-02T11:33:44.746493+0000","last_undegraded":"2024-04-02T11:33:44.746493+0000","last_fullsized":"2024-04-02T11:33:44.746493+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.476796+0000","last_change":"2024-04-02T11:32:04.531627+0000","last_active":"2024-04-02T11:33:44.476796+0000","last_peered":"2024-04-02T11:33:44.476796+0000","last_clean":"2024-04-02T11:33:44.476796+0000","last_became_active":"2024-04-02T11:32:04.523227+0000","last_became_peered":"2024-04-02T11:32:04.523227+0000","last_unstale":"2024-04-02T11:33:44.476796+0000","last_undegraded":"2024-04-02T11:33:44.476796+0000","last_fullsized":"2024-04-02T11:33:44.476796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.5","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.613522+0000","last_change":"2024-04-02T11:33:41.243595+0000","last_active":"2024-04-02T11:33:44.613522+0000","last_peered":"2024-04-02T11:33:44.613522+0000","last_clean":"2024-04-02T11:33:44.613522+0000","last_became_active":"2024-04-02T11:33:41.229522+0000","last_became_peered":"2024-04-02T11:33:41.229522+0000","last_unstale":"2024-04-02T11:33:44.613522+0000","last_undegraded":"2024-04-02T11:33:44.613522+0000","last_fullsized":"2024-04-02T11:33:44.613522+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.508048+0000","last_change":"2024-04-02T11:32:04.499203+0000","last_active":"2024-04-02T11:33:44.508048+0000","last_peered":"2024-04-02T11:33:44.508048+0000","last_clean":"2024-04-02T11:33:44.508048+0000","last_became_active":"2024-04-02T11:32:04.490360+0000","last_became_peered":"2024-04-02T11:32:04.490360+0000","last_unstale":"2024-04-02T11:33:44.508048+0000","last_undegraded":"2024-04-02T11:33:44.508048+0000","last_fullsized":"2024-04-02T11:33:44.508048+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.619749+0000","last_change":"2024-04-02T11:33:41.308637+0000","last_active":"2024-04-02T11:33:44.619749+0000","last_peered":"2024-04-02T11:33:44.619749+0000","last_clean":"2024-04-02T11:33:44.619749+0000","last_became_active":"2024-04-02T11:33:41.301597+0000","last_became_peered":"2024-04-02T11:33:41.301597+0000","last_unstale":"2024-04-02T11:33:44.619749+0000","last_undegraded":"2024-04-02T11:33:44.619749+0000","last_fullsized":"2024-04-02T11:33:44.619749+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.751562+0000","last_change":"2024-04-02T11:32:04.460524+0000","last_active":"2024-04-02T11:33:44.751562+0000","last_peered":"2024-04-02T11:33:44.751562+0000","last_clean":"2024-04-02T11:33:44.751562+0000","last_became_active":"2024-04-02T11:32:04.427391+0000","last_became_peered":"2024-04-02T11:32:04.427391+0000","last_unstale":"2024-04-02T11:33:44.751562+0000","last_undegraded":"2024-04-02T11:33:44.751562+0000","last_fullsized":"2024-04-02T11:33:44.751562+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.087235+0000","last_change":"2024-04-02T11:33:41.282206+0000","last_active":"2024-04-02T11:33:44.087235+0000","last_peered":"2024-04-02T11:33:44.087235+0000","last_clean":"2024-04-02T11:33:44.087235+0000","last_became_active":"2024-04-02T11:33:41.264133+0000","last_became_peered":"2024-04-02T11:33:41.264133+0000","last_unstale":"2024-04-02T11:33:44.087235+0000","last_undegraded":"2024-04-02T11:33:44.087235+0000","last_fullsized":"2024-04-02T11:33:44.087235+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.618701+0000","last_change":"2024-04-02T11:32:04.403716+0000","last_active":"2024-04-02T11:33:44.618701+0000","last_peered":"2024-04-02T11:33:44.618701+0000","last_clean":"2024-04-02T11:33:44.618701+0000","last_became_active":"2024-04-02T11:32:04.390779+0000","last_became_peered":"2024-04-02T11:32:04.390779+0000","last_unstale":"2024-04-02T11:33:44.618701+0000","last_undegraded":"2024-04-02T11:33:44.618701+0000","last_fullsized":"2024-04-02T11:33:44.618701+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.8","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.743680+0000","last_change":"2024-04-02T11:33:41.322754+0000","last_active":"2024-04-02T11:33:44.743680+0000","last_peered":"2024-04-02T11:33:44.743680+0000","last_clean":"2024-04-02T11:33:44.743680+0000","last_became_active":"2024-04-02T11:33:41.311852+0000","last_became_peered":"2024-04-02T11:33:41.311852+0000","last_unstale":"2024-04-02T11:33:44.743680+0000","last_undegraded":"2024-04-02T11:33:44.743680+0000","last_fullsized":"2024-04-02T11:33:44.743680+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.749745+0000","last_change":"2024-04-02T11:32:04.441727+0000","last_active":"2024-04-02T11:33:44.749745+0000","last_peered":"2024-04-02T11:33:44.749745+0000","last_clean":"2024-04-02T11:33:44.749745+0000","last_became_active":"2024-04-02T11:32:04.409646+0000","last_became_peered":"2024-04-02T11:32:04.409646+0000","last_unstale":"2024-04-02T11:33:44.749745+0000","last_undegraded":"2024-04-02T11:33:44.749745+0000","last_fullsized":"2024-04-02T11:33:44.749745+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.9","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773128+0000","last_change":"2024-04-02T11:33:41.293468+0000","last_active":"2024-04-02T11:33:44.773128+0000","last_peered":"2024-04-02T11:33:44.773128+0000","last_clean":"2024-04-02T11:33:44.773128+0000","last_became_active":"2024-04-02T11:33:41.267676+0000","last_became_peered":"2024-04-02T11:33:41.267676+0000","last_unstale":"2024-04-02T11:33:44.773128+0000","last_undegraded":"2024-04-02T11:33:44.773128+0000","last_fullsized":"2024-04-02T11:33:44.773128+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.509804+0000","last_change":"2024-04-02T11:32:04.404984+0000","last_active":"2024-04-02T11:33:44.509804+0000","last_peered":"2024-04-02T11:33:44.509804+0000","last_clean":"2024-04-02T11:33:44.509804+0000","last_became_active":"2024-04-02T11:32:04.387697+0000","last_became_peered":"2024-04-02T11:32:04.387697+0000","last_unstale":"2024-04-02T11:33:44.509804+0000","last_undegraded":"2024-04-02T11:33:44.509804+0000","last_fullsized":"2024-04-02T11:33:44.509804+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.755181+0000","last_change":"2024-04-02T11:33:41.296184+0000","last_active":"2024-04-02T11:33:44.755181+0000","last_peered":"2024-04-02T11:33:44.755181+0000","last_clean":"2024-04-02T11:33:44.755181+0000","last_became_active":"2024-04-02T11:33:41.271175+0000","last_became_peered":"2024-04-02T11:33:41.271175+0000","last_unstale":"2024-04-02T11:33:44.755181+0000","last_undegraded":"2024-04-02T11:33:44.755181+0000","last_fullsized":"2024-04-02T11:33:44.755181+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.732609+0000","last_change":"2024-04-02T11:32:04.423708+0000","last_active":"2024-04-02T11:33:44.732609+0000","last_peered":"2024-04-02T11:33:44.732609+0000","last_clean":"2024-04-02T11:33:44.732609+0000","last_became_active":"2024-04-02T11:32:04.401780+0000","last_became_peered":"2024-04-02T11:32:04.401780+0000","last_unstale":"2024-04-02T11:33:44.732609+0000","last_undegraded":"2024-04-02T11:33:44.732609+0000","last_fullsized":"2024-04-02T11:33:44.732609+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.742027+0000","last_change":"2024-04-02T11:33:41.274959+0000","last_active":"2024-04-02T11:33:44.742027+0000","last_peered":"2024-04-02T11:33:44.742027+0000","last_clean":"2024-04-02T11:33:44.742027+0000","last_became_active":"2024-04-02T11:33:41.254717+0000","last_became_peered":"2024-04-02T11:33:41.254717+0000","last_unstale":"2024-04-02T11:33:44.742027+0000","last_undegraded":"2024-04-02T11:33:44.742027+0000","last_fullsized":"2024-04-02T11:33:44.742027+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.728056+0000","last_change":"2024-04-02T11:32:04.457456+0000","last_active":"2024-04-02T11:33:44.728056+0000","last_peered":"2024-04-02T11:33:44.728056+0000","last_clean":"2024-04-02T11:33:44.728056+0000","last_became_active":"2024-04-02T11:32:04.436936+0000","last_became_peered":"2024-04-02T11:32:04.436936+0000","last_unstale":"2024-04-02T11:33:44.728056+0000","last_undegraded":"2024-04-02T11:33:44.728056+0000","last_fullsized":"2024-04-02T11:33:44.728056+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.579024+0000","last_change":"2024-04-02T11:33:41.259181+0000","last_active":"2024-04-02T11:33:44.579024+0000","last_peered":"2024-04-02T11:33:44.579024+0000","last_clean":"2024-04-02T11:33:44.579024+0000","last_became_active":"2024-04-02T11:33:41.249228+0000","last_became_peered":"2024-04-02T11:33:41.249228+0000","last_unstale":"2024-04-02T11:33:44.579024+0000","last_undegraded":"2024-04-02T11:33:44.579024+0000","last_fullsized":"2024-04-02T11:33:44.579024+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.503670+0000","last_change":"2024-04-02T11:32:04.429226+0000","last_active":"2024-04-02T11:33:44.503670+0000","last_peered":"2024-04-02T11:33:44.503670+0000","last_clean":"2024-04-02T11:33:44.503670+0000","last_became_active":"2024-04-02T11:32:04.407776+0000","last_became_peered":"2024-04-02T11:32:04.407776+0000","last_unstale":"2024-04-02T11:33:44.503670+0000","last_undegraded":"2024-04-02T11:33:44.503670+0000","last_fullsized":"2024-04-02T11:33:44.503670+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.480198+0000","last_change":"2024-04-02T11:33:41.329049+0000","last_active":"2024-04-02T11:33:44.480198+0000","last_peered":"2024-04-02T11:33:44.480198+0000","last_clean":"2024-04-02T11:33:44.480198+0000","last_became_active":"2024-04-02T11:33:41.323281+0000","last_became_peered":"2024-04-02T11:33:41.323281+0000","last_unstale":"2024-04-02T11:33:44.480198+0000","last_undegraded":"2024-04-02T11:33:44.480198+0000","last_fullsized":"2024-04-02T11:33:44.480198+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.494895+0000","last_change":"2024-04-02T11:32:04.558136+0000","last_active":"2024-04-02T11:33:44.494895+0000","last_peered":"2024-04-02T11:33:44.494895+0000","last_clean":"2024-04-02T11:33:44.494895+0000","last_became_active":"2024-04-02T11:32:04.551595+0000","last_became_peered":"2024-04-02T11:32:04.551595+0000","last_unstale":"2024-04-02T11:33:44.494895+0000","last_undegraded":"2024-04-02T11:33:44.494895+0000","last_fullsized":"2024-04-02T11:33:44.494895+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.12","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.478405+0000","last_change":"2024-04-02T11:33:41.281874+0000","last_active":"2024-04-02T11:33:44.478405+0000","last_peered":"2024-04-02T11:33:44.478405+0000","last_clean":"2024-04-02T11:33:44.478405+0000","last_became_active":"2024-04-02T11:33:41.274203+0000","last_became_peered":"2024-04-02T11:33:41.274203+0000","last_unstale":"2024-04-02T11:33:44.478405+0000","last_undegraded":"2024-04-02T11:33:44.478405+0000","last_fullsized":"2024-04-02T11:33:44.478405+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.485930+0000","last_change":"2024-04-02T11:32:04.417231+0000","last_active":"2024-04-02T11:33:44.485930+0000","last_peered":"2024-04-02T11:33:44.485930+0000","last_clean":"2024-04-02T11:33:44.485930+0000","last_became_active":"2024-04-02T11:32:04.396181+0000","last_became_peered":"2024-04-02T11:32:04.396181+0000","last_unstale":"2024-04-02T11:33:44.485930+0000","last_undegraded":"2024-04-02T11:33:44.485930+0000","last_fullsized":"2024-04-02T11:33:44.485930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.13","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.602064+0000","last_change":"2024-04-02T11:33:41.245713+0000","last_active":"2024-04-02T11:33:44.602064+0000","last_peered":"2024-04-02T11:33:44.602064+0000","last_clean":"2024-04-02T11:33:44.602064+0000","last_became_active":"2024-04-02T11:33:41.226115+0000","last_became_peered":"2024-04-02T11:33:41.226115+0000","last_unstale":"2024-04-02T11:33:44.602064+0000","last_undegraded":"2024-04-02T11:33:44.602064+0000","last_fullsized":"2024-04-02T11:33:44.602064+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.754146+0000","last_change":"2024-04-02T11:32:04.450092+0000","last_active":"2024-04-02T11:33:44.754146+0000","last_peered":"2024-04-02T11:33:44.754146+0000","last_clean":"2024-04-02T11:33:44.754146+0000","last_became_active":"2024-04-02T11:32:04.418023+0000","last_became_peered":"2024-04-02T11:32:04.418023+0000","last_unstale":"2024-04-02T11:33:44.754146+0000","last_undegraded":"2024-04-02T11:33:44.754146+0000","last_fullsized":"2024-04-02T11:33:44.754146+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.10","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.572668+0000","last_change":"2024-04-02T11:33:41.266477+0000","last_active":"2024-04-02T11:33:44.572668+0000","last_peered":"2024-04-02T11:33:44.572668+0000","last_clean":"2024-04-02T11:33:44.572668+0000","last_became_active":"2024-04-02T11:33:41.256091+0000","last_became_peered":"2024-04-02T11:33:41.256091+0000","last_unstale":"2024-04-02T11:33:44.572668+0000","last_undegraded":"2024-04-02T11:33:44.572668+0000","last_fullsized":"2024-04-02T11:33:44.572668+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.611376+0000","last_change":"2024-04-02T11:32:04.397771+0000","last_active":"2024-04-02T11:33:44.611376+0000","last_peered":"2024-04-02T11:33:44.611376+0000","last_clean":"2024-04-02T11:33:44.611376+0000","last_became_active":"2024-04-02T11:32:04.378859+0000","last_became_peered":"2024-04-02T11:32:04.378859+0000","last_unstale":"2024-04-02T11:33:44.611376+0000","last_undegraded":"2024-04-02T11:33:44.611376+0000","last_fullsized":"2024-04-02T11:33:44.611376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.11","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.758547+0000","last_change":"2024-04-02T11:33:41.263642+0000","last_active":"2024-04-02T11:33:44.758547+0000","last_peered":"2024-04-02T11:33:44.758547+0000","last_clean":"2024-04-02T11:33:44.758547+0000","last_became_active":"2024-04-02T11:33:41.250607+0000","last_became_peered":"2024-04-02T11:33:41.250607+0000","last_unstale":"2024-04-02T11:33:44.758547+0000","last_undegraded":"2024-04-02T11:33:44.758547+0000","last_fullsized":"2024-04-02T11:33:44.758547+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.073706+0000","last_change":"2024-04-02T11:32:04.463785+0000","last_active":"2024-04-02T11:33:44.073706+0000","last_peered":"2024-04-02T11:33:44.073706+0000","last_clean":"2024-04-02T11:33:44.073706+0000","last_became_active":"2024-04-02T11:32:04.457870+0000","last_became_peered":"2024-04-02T11:32:04.457870+0000","last_unstale":"2024-04-02T11:33:44.073706+0000","last_undegraded":"2024-04-02T11:33:44.073706+0000","last_fullsized":"2024-04-02T11:33:44.073706+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.16","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.605469+0000","last_change":"2024-04-02T11:33:41.241536+0000","last_active":"2024-04-02T11:33:44.605469+0000","last_peered":"2024-04-02T11:33:44.605469+0000","last_clean":"2024-04-02T11:33:44.605469+0000","last_became_active":"2024-04-02T11:33:41.232699+0000","last_became_peered":"2024-04-02T11:33:41.232699+0000","last_unstale":"2024-04-02T11:33:44.605469+0000","last_undegraded":"2024-04-02T11:33:44.605469+0000","last_fullsized":"2024-04-02T11:33:44.605469+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.071157+0000","last_change":"2024-04-02T11:32:04.384861+0000","last_active":"2024-04-02T11:33:44.071157+0000","last_peered":"2024-04-02T11:33:44.071157+0000","last_clean":"2024-04-02T11:33:44.071157+0000","last_became_active":"2024-04-02T11:32:04.368304+0000","last_became_peered":"2024-04-02T11:32:04.368304+0000","last_unstale":"2024-04-02T11:33:44.071157+0000","last_undegraded":"2024-04-02T11:33:44.071157+0000","last_fullsized":"2024-04-02T11:33:44.071157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.17","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.590092+0000","last_change":"2024-04-02T11:33:41.281940+0000","last_active":"2024-04-02T11:33:44.590092+0000","last_peered":"2024-04-02T11:33:44.590092+0000","last_clean":"2024-04-02T11:33:44.590092+0000","last_became_active":"2024-04-02T11:33:41.275074+0000","last_became_peered":"2024-04-02T11:33:41.275074+0000","last_unstale":"2024-04-02T11:33:44.590092+0000","last_undegraded":"2024-04-02T11:33:44.590092+0000","last_fullsized":"2024-04-02T11:33:44.590092+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.091094+0000","last_change":"2024-04-02T11:32:04.377416+0000","last_active":"2024-04-02T11:33:44.091094+0000","last_peered":"2024-04-02T11:33:44.091094+0000","last_clean":"2024-04-02T11:33:44.091094+0000","last_became_active":"2024-04-02T11:32:04.364464+0000","last_became_peered":"2024-04-02T11:32:04.364464+0000","last_unstale":"2024-04-02T11:33:44.091094+0000","last_undegraded":"2024-04-02T11:33:44.091094+0000","last_fullsized":"2024-04-02T11:33:44.091094+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.14","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.076833+0000","last_change":"2024-04-02T11:33:41.311816+0000","last_active":"2024-04-02T11:33:44.076833+0000","last_peered":"2024-04-02T11:33:44.076833+0000","last_clean":"2024-04-02T11:33:44.076833+0000","last_became_active":"2024-04-02T11:33:41.290381+0000","last_became_peered":"2024-04-02T11:33:41.290381+0000","last_unstale":"2024-04-02T11:33:44.076833+0000","last_undegraded":"2024-04-02T11:33:44.076833+0000","last_fullsized":"2024-04-02T11:33:44.076833+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.599260+0000","last_change":"2024-04-02T11:32:04.407271+0000","last_active":"2024-04-02T11:33:44.599260+0000","last_peered":"2024-04-02T11:33:44.599260+0000","last_clean":"2024-04-02T11:33:44.599260+0000","last_became_active":"2024-04-02T11:32:04.386636+0000","last_became_peered":"2024-04-02T11:32:04.386636+0000","last_unstale":"2024-04-02T11:33:44.599260+0000","last_undegraded":"2024-04-02T11:33:44.599260+0000","last_fullsized":"2024-04-02T11:33:44.599260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.15","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.085724+0000","last_change":"2024-04-02T11:33:41.299177+0000","last_active":"2024-04-02T11:33:44.085724+0000","last_peered":"2024-04-02T11:33:44.085724+0000","last_clean":"2024-04-02T11:33:44.085724+0000","last_became_active":"2024-04-02T11:33:41.271590+0000","last_became_peered":"2024-04-02T11:33:41.271590+0000","last_unstale":"2024-04-02T11:33:44.085724+0000","last_undegraded":"2024-04-02T11:33:44.085724+0000","last_fullsized":"2024-04-02T11:33:44.085724+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.703445+0000","last_change":"2024-04-02T11:32:04.421594+0000","last_active":"2024-04-02T11:33:44.703445+0000","last_peered":"2024-04-02T11:33:44.703445+0000","last_clean":"2024-04-02T11:33:44.703445+0000","last_became_active":"2024-04-02T11:32:04.404970+0000","last_became_peered":"2024-04-02T11:32:04.404970+0000","last_unstale":"2024-04-02T11:33:44.703445+0000","last_undegraded":"2024-04-02T11:33:44.703445+0000","last_fullsized":"2024-04-02T11:33:44.703445+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113166+0000","last_change":"2024-04-02T11:33:41.294167+0000","last_active":"2024-04-02T11:33:44.113166+0000","last_peered":"2024-04-02T11:33:44.113166+0000","last_clean":"2024-04-02T11:33:44.113166+0000","last_became_active":"2024-04-02T11:33:41.278749+0000","last_became_peered":"2024-04-02T11:33:41.278749+0000","last_unstale":"2024-04-02T11:33:44.113166+0000","last_undegraded":"2024-04-02T11:33:44.113166+0000","last_fullsized":"2024-04-02T11:33:44.113166+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.753274+0000","last_change":"2024-04-02T11:32:04.439591+0000","last_active":"2024-04-02T11:33:44.753274+0000","last_peered":"2024-04-02T11:33:44.753274+0000","last_clean":"2024-04-02T11:33:44.753274+0000","last_became_active":"2024-04-02T11:32:04.414684+0000","last_became_peered":"2024-04-02T11:32:04.414684+0000","last_unstale":"2024-04-02T11:33:44.753274+0000","last_undegraded":"2024-04-02T11:33:44.753274+0000","last_fullsized":"2024-04-02T11:33:44.753274+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.739455+0000","last_change":"2024-04-02T11:33:41.277360+0000","last_active":"2024-04-02T11:33:44.739455+0000","last_peered":"2024-04-02T11:33:44.739455+0000","last_clean":"2024-04-02T11:33:44.739455+0000","last_became_active":"2024-04-02T11:33:41.258789+0000","last_became_peered":"2024-04-02T11:33:41.258789+0000","last_unstale":"2024-04-02T11:33:44.739455+0000","last_undegraded":"2024-04-02T11:33:44.739455+0000","last_fullsized":"2024-04-02T11:33:44.739455+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.510847+0000","last_change":"2024-04-02T11:32:04.419077+0000","last_active":"2024-04-02T11:33:44.510847+0000","last_peered":"2024-04-02T11:33:44.510847+0000","last_clean":"2024-04-02T11:33:44.510847+0000","last_became_active":"2024-04-02T11:32:04.392023+0000","last_became_peered":"2024-04-02T11:32:04.392023+0000","last_unstale":"2024-04-02T11:33:44.510847+0000","last_undegraded":"2024-04-02T11:33:44.510847+0000","last_fullsized":"2024-04-02T11:33:44.510847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.18","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.501017+0000","last_change":"2024-04-02T11:33:41.379485+0000","last_active":"2024-04-02T11:33:44.501017+0000","last_peered":"2024-04-02T11:33:44.501017+0000","last_clean":"2024-04-02T11:33:44.501017+0000","last_became_active":"2024-04-02T11:33:41.374309+0000","last_became_peered":"2024-04-02T11:33:41.374309+0000","last_unstale":"2024-04-02T11:33:44.501017+0000","last_undegraded":"2024-04-02T11:33:44.501017+0000","last_fullsized":"2024-04-02T11:33:44.501017+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.766807+0000","last_change":"2024-04-02T11:32:04.464793+0000","last_active":"2024-04-02T11:33:44.766807+0000","last_peered":"2024-04-02T11:33:44.766807+0000","last_clean":"2024-04-02T11:33:44.766807+0000","last_became_active":"2024-04-02T11:32:04.445170+0000","last_became_peered":"2024-04-02T11:32:04.445170+0000","last_unstale":"2024-04-02T11:33:44.766807+0000","last_undegraded":"2024-04-02T11:33:44.766807+0000","last_fullsized":"2024-04-02T11:33:44.766807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.19","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.493963+0000","last_change":"2024-04-02T11:33:41.353471+0000","last_active":"2024-04-02T11:33:44.493963+0000","last_peered":"2024-04-02T11:33:44.493963+0000","last_clean":"2024-04-02T11:33:44.493963+0000","last_became_active":"2024-04-02T11:33:41.347559+0000","last_became_peered":"2024-04-02T11:33:41.347559+0000","last_unstale":"2024-04-02T11:33:44.493963+0000","last_undegraded":"2024-04-02T11:33:44.493963+0000","last_fullsized":"2024-04-02T11:33:44.493963+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.492318+0000","last_change":"2024-04-02T11:32:04.474632+0000","last_active":"2024-04-02T11:33:44.492318+0000","last_peered":"2024-04-02T11:33:44.492318+0000","last_clean":"2024-04-02T11:33:44.492318+0000","last_became_active":"2024-04-02T11:32:04.467593+0000","last_became_peered":"2024-04-02T11:32:04.467593+0000","last_unstale":"2024-04-02T11:33:44.492318+0000","last_undegraded":"2024-04-02T11:33:44.492318+0000","last_fullsized":"2024-04-02T11:33:44.492318+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.765959+0000","last_change":"2024-04-02T11:33:41.303867+0000","last_active":"2024-04-02T11:33:44.765959+0000","last_peered":"2024-04-02T11:33:44.765959+0000","last_clean":"2024-04-02T11:33:44.765959+0000","last_became_active":"2024-04-02T11:33:41.282720+0000","last_became_peered":"2024-04-02T11:33:41.282720+0000","last_unstale":"2024-04-02T11:33:44.765959+0000","last_undegraded":"2024-04-02T11:33:44.765959+0000","last_fullsized":"2024-04-02T11:33:44.765959+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.489009+0000","last_change":"2024-04-02T11:32:04.415345+0000","last_active":"2024-04-02T11:33:44.489009+0000","last_peered":"2024-04-02T11:33:44.489009+0000","last_clean":"2024-04-02T11:33:44.489009+0000","last_became_active":"2024-04-02T11:32:04.400375+0000","last_became_peered":"2024-04-02T11:32:04.400375+0000","last_unstale":"2024-04-02T11:33:44.489009+0000","last_undegraded":"2024-04-02T11:33:44.489009+0000","last_fullsized":"2024-04-02T11:33:44.489009+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.777113+0000","last_change":"2024-04-02T11:33:41.334657+0000","last_active":"2024-04-02T11:33:44.777113+0000","last_peered":"2024-04-02T11:33:44.777113+0000","last_clean":"2024-04-02T11:33:44.777113+0000","last_became_active":"2024-04-02T11:33:41.325633+0000","last_became_peered":"2024-04-02T11:33:41.325633+0000","last_unstale":"2024-04-02T11:33:44.777113+0000","last_undegraded":"2024-04-02T11:33:44.777113+0000","last_fullsized":"2024-04-02T11:33:44.777113+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.471747+0000","last_change":"2024-04-02T11:32:04.461967+0000","last_active":"2024-04-02T11:33:44.471747+0000","last_peered":"2024-04-02T11:33:44.471747+0000","last_clean":"2024-04-02T11:33:44.471747+0000","last_became_active":"2024-04-02T11:32:04.456013+0000","last_became_peered":"2024-04-02T11:32:04.456013+0000","last_unstale":"2024-04-02T11:33:44.471747+0000","last_undegraded":"2024-04-02T11:33:44.471747+0000","last_fullsized":"2024-04-02T11:33:44.471747+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.506496+0000","last_change":"2024-04-02T11:33:41.285447+0000","last_active":"2024-04-02T11:33:44.506496+0000","last_peered":"2024-04-02T11:33:44.506496+0000","last_clean":"2024-04-02T11:33:44.506496+0000","last_became_active":"2024-04-02T11:33:41.270749+0000","last_became_peered":"2024-04-02T11:33:41.270749+0000","last_unstale":"2024-04-02T11:33:44.506496+0000","last_undegraded":"2024-04-02T11:33:44.506496+0000","last_fullsized":"2024-04-02T11:33:44.506496+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":151,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.074943+0000","last_change":"2024-04-02T11:32:04.431094+0000","last_active":"2024-04-02T11:33:44.074943+0000","last_peered":"2024-04-02T11:33:44.074943+0000","last_clean":"2024-04-02T11:33:44.074943+0000","last_became_active":"2024-04-02T11:32:04.423058+0000","last_became_peered":"2024-04-02T11:32:04.423058+0000","last_unstale":"2024-04-02T11:33:44.074943+0000","last_undegraded":"2024-04-02T11:33:44.074943+0000","last_fullsized":"2024-04-02T11:33:44.074943+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.1d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.106087+0000","last_change":"2024-04-02T11:33:41.274565+0000","last_active":"2024-04-02T11:33:44.106087+0000","last_peered":"2024-04-02T11:33:44.106087+0000","last_clean":"2024-04-02T11:33:44.106087+0000","last_became_active":"2024-04-02T11:33:41.258583+0000","last_became_peered":"2024-04-02T11:33:41.258583+0000","last_unstale":"2024-04-02T11:33:44.106087+0000","last_undegraded":"2024-04-02T11:33:44.106087+0000","last_fullsized":"2024-04-02T11:33:44.106087+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.593632+0000","last_change":"2024-04-02T11:32:04.401482+0000","last_active":"2024-04-02T11:33:44.593632+0000","last_peered":"2024-04-02T11:33:44.593632+0000","last_clean":"2024-04-02T11:33:44.593632+0000","last_became_active":"2024-04-02T11:32:04.383344+0000","last_became_peered":"2024-04-02T11:32:04.383344+0000","last_unstale":"2024-04-02T11:33:44.593632+0000","last_undegraded":"2024-04-02T11:33:44.593632+0000","last_fullsized":"2024-04-02T11:33:44.593632+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.069239+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:33:44.069239+0000","last_peered":"2024-04-02T11:33:44.069239+0000","last_clean":"2024-04-02T11:33:44.069239+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:33:44.069239+0000","last_undegraded":"2024-04-02T11:33:44.069239+0000","last_fullsized":"2024-04-02T11:33:44.069239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.627702+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:33:44.627702+0000","last_peered":"2024-04-02T11:33:44.627702+0000","last_clean":"2024-04-02T11:33:44.627702+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:33:44.627702+0000","last_undegraded":"2024-04-02T11:33:44.627702+0000","last_fullsized":"2024-04-02T11:33:44.627702+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":741,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.722011+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:33:44.722011+0000","last_peered":"2024-04-02T11:33:44.722011+0000","last_clean":"2024-04-02T11:33:44.722011+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:33:44.722011+0000","last_undegraded":"2024-04-02T11:33:44.722011+0000","last_fullsized":"2024-04-02T11:33:44.722011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.623600+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:33:44.623600+0000","last_peered":"2024-04-02T11:33:44.623600+0000","last_clean":"2024-04-02T11:33:44.623600+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:33:44.623600+0000","last_undegraded":"2024-04-02T11:33:44.623600+0000","last_fullsized":"2024-04-02T11:33:44.623600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.473736+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:33:44.473736+0000","last_peered":"2024-04-02T11:33:44.473736+0000","last_clean":"2024-04-02T11:33:44.473736+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:33:44.473736+0000","last_undegraded":"2024-04-02T11:33:44.473736+0000","last_fullsized":"2024-04-02T11:33:44.473736+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.110855+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:33:44.110855+0000","last_peered":"2024-04-02T11:33:44.110855+0000","last_clean":"2024-04-02T11:33:44.110855+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:33:44.110855+0000","last_undegraded":"2024-04-02T11:33:44.110855+0000","last_fullsized":"2024-04-02T11:33:44.110855+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.504908+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:33:44.504908+0000","last_peered":"2024-04-02T11:33:44.504908+0000","last_clean":"2024-04-02T11:33:44.504908+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:33:44.504908+0000","last_undegraded":"2024-04-02T11:33:44.504908+0000","last_fullsized":"2024-04-02T11:33:44.504908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113298+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:33:44.113298+0000","last_peered":"2024-04-02T11:33:44.113298+0000","last_clean":"2024-04-02T11:33:44.113298+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:33:44.113298+0000","last_undegraded":"2024-04-02T11:33:44.113298+0000","last_fullsized":"2024-04-02T11:33:44.113298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":877,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.734120+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:33:44.734120+0000","last_peered":"2024-04-02T11:33:44.734120+0000","last_clean":"2024-04-02T11:33:44.734120+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:33:44.734120+0000","last_undegraded":"2024-04-02T11:33:44.734120+0000","last_fullsized":"2024-04-02T11:33:44.734120+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":86,"num_pg":32,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":4},{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":31104,"kb_avail":104824672,"statfs":{"total":107374182400,"available":107340464128,"internally_reserved":0,"allocated":1859584,"data_stored":1037844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31850496},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829396,"statfs":{"total":107374182400,"available":107345301504,"internally_reserved":0,"allocated":1282048,"data_stored":451769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":54576,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104803024,"statfs":{"total":107374182400,"available":107318296576,"internally_reserved":0,"allocated":1286144,"data_stored":456057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":55140,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104802460,"statfs":{"total":107374182400,"available":107317719040,"internally_reserved":0,"allocated":1863680,"data_stored":1037972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:28:00.844 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-02T14:28:00.844 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:28:00.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.955+0000 7f2e13cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.959+0000 7f2e13cad640 1 Processor -- start 2024-04-02T14:28:00.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.960+0000 7f2e13cad640 1 -- start start 2024-04-02T14:28:00.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e13cad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2e0c0703a0 0x7f2e0c1316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e13cad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0c073340 con 0x7f2e0c131c10 2024-04-02T14:28:00.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e13cad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c0a48e0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11a49640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2e0c0703a0 0x7f2e0c1316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11248640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2e0c131c10 0x7f2e0c134740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36994/0 (socket says 172.21.15.133:36994) 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11248640 1 -- 172.21.15.133:0/525643729 learned_addr learned my addr 172.21.15.133:0/525643729 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11a49640 1 -- 172.21.15.133:0/525643729 >> v1:172.21.15.133:6789/0 conn(0x7f2e0c131c10 legacy=0x7f2e0c134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11a49640 1 -- 172.21.15.133:0/525643729 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c0707b0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e11a49640 1 --2- 172.21.15.133:0/525643729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1316d0 crc :-1 s=READY pgs=12314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292c9b99ba853467 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e10a47640 1 -- 172.21.15.133:0/525643729 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e08009d90 con 0x7f2e0c0703a0 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e10a47640 1 -- 172.21.15.133:0/525643729 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e0800e230 con 0x7f2e0c0703a0 2024-04-02T14:28:00.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.961+0000 7f2e10a47640 1 -- 172.21.15.133:0/525643729 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e0800e4d0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 -- 172.21.15.133:0/525643729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 msgr2=0x7f2e0c1316d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:00.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 --2- 172.21.15.133:0/525643729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1316d0 crc :-1 s=READY pgs=12314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 -- 172.21.15.133:0/525643729 shutdown_connections 2024-04-02T14:28:00.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 --2- 172.21.15.133:0/525643729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1316d0 unknown :-1 s=CLOSED pgs=12314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 -- 172.21.15.133:0/525643729 >> 172.21.15.133:0/525643729 conn(0x7f2e0c0a2920 msgr2=0x7f2e0c0a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 -- 172.21.15.133:0/525643729 shutdown_connections 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 1 -- 172.21.15.133:0/525643729 wait complete. 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.962+0000 7f2e13cad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.964+0000 7f2e13cad640 1 Processor -- start 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.964+0000 7f2e13cad640 1 -- start start 2024-04-02T14:28:00.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e13cad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e13cad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c11f520 con 0x7f2e0c0703a0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e11a49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e11a49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47742/0 (socket says 172.21.15.133:47742) 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e11a49640 1 -- 172.21.15.133:0/3647554135 learned_addr learned my addr 172.21.15.133:0/3647554135 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e11a49640 1 -- 172.21.15.133:0/3647554135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1321a0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e11a49640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 crc :-1 s=READY pgs=12315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e08014620 con 0x7f2e0c0703a0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e08014780 con 0x7f2e0c0703a0 2024-04-02T14:28:00.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c11e1e0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e08014a20 con 0x7f2e0c0703a0 2024-04-02T14:28:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.965+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c11f810 con 0x7f2e0c0703a0 2024-04-02T14:28:00.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.966+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f2e08014bd0 con 0x7f2e0c0703a0 2024-04-02T14:28:00.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.966+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c063400 con 0x7f2e0c0703a0 2024-04-02T14:28:00.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.966+0000 7f2e027fc640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 0x7f2df003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:00.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.966+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7f2e08019040 con 0x7f2e0c0703a0 2024-04-02T14:28:00.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.967+0000 7f2e11248640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 0x7f2df003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:00.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.967+0000 7f2e11248640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 0x7f2df003ae90 crc :-1 s=READY pgs=3800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:00.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:00.972+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f2e08018190 con 0x7f2e0c0703a0 2024-04-02T14:28:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.146+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e0c070ac0 con 0x7f2df0038980 2024-04-02T14:28:01.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.151+0000 7f2e027fc640 1 -- 172.21.15.133:0/3647554135 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+187053 (crc 0 0 0) 0x7f2dfc005910 con 0x7f2df0038980 2024-04-02T14:28:01.155 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:01.156 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T14:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.153+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 msgr2=0x7f2df003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.153+0000 7f2e13cad640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 0x7f2df003ae90 crc :-1 s=READY pgs=3800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.153+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 msgr2=0x7f2e0c1c9c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.153+0000 7f2e13cad640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 crc :-1 s=READY pgs=12315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 shutdown_connections 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f2df0038980 0x7f2df003ae90 unknown :-1 s=CLOSED pgs=3800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 --2- 172.21.15.133:0/3647554135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e0c0703a0 0x7f2e0c1c9c10 unknown :-1 s=CLOSED pgs=12315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 >> 172.21.15.133:0/3647554135 conn(0x7f2e0c0a2920 msgr2=0x7f2e0c09ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 shutdown_connections 2024-04-02T14:28:01.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.154+0000 7f2e13cad640 1 -- 172.21.15.133:0/3647554135 wait complete. 2024-04-02T14:28:01.166 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5713,"stamp":"2024-04-02T14:27:59.854995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594469,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":145,"num_write_kb":1905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":170848,"kb_used_data":6144,"kb_used_omap":0,"kb_used_meta":164672,"kb_avail":419259552,"statfs":{"total":429496729600,"available":429321781248,"internally_reserved":0,"allocated":6291456,"data_stored":2983642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":168624128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"86.2","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.060011+0000","last_change":"2024-04-02T11:33:41.348309+0000","last_active":"2024-04-02T11:33:44.060011+0000","last_peered":"2024-04-02T11:33:44.060011+0000","last_clean":"2024-04-02T11:33:44.060011+0000","last_became_active":"2024-04-02T11:33:41.342906+0000","last_became_peered":"2024-04-02T11:33:41.342906+0000","last_unstale":"2024-04-02T11:33:44.060011+0000","last_undegraded":"2024-04-02T11:33:44.060011+0000","last_fullsized":"2024-04-02T11:33:44.060011+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1f","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.083353+0000","last_change":"2024-04-02T11:32:04.380690+0000","last_active":"2024-04-02T11:33:44.083353+0000","last_peered":"2024-04-02T11:33:44.083353+0000","last_clean":"2024-04-02T11:33:44.083353+0000","last_became_active":"2024-04-02T11:32:04.372115+0000","last_became_peered":"2024-04-02T11:32:04.372115+0000","last_unstale":"2024-04-02T11:33:44.083353+0000","last_undegraded":"2024-04-02T11:33:44.083353+0000","last_fullsized":"2024-04-02T11:33:44.083353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.3","version":"285'1","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.111981+0000","last_change":"2024-04-02T11:33:41.302524+0000","last_active":"2024-04-02T11:33:44.111981+0000","last_peered":"2024-04-02T11:33:44.111981+0000","last_clean":"2024-04-02T11:33:44.111981+0000","last_became_active":"2024-04-02T11:33:41.286642+0000","last_became_peered":"2024-04-02T11:33:41.286642+0000","last_unstale":"2024-04-02T11:33:44.111981+0000","last_undegraded":"2024-04-02T11:33:44.111981+0000","last_fullsized":"2024-04-02T11:33:44.111981+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.479267+0000","last_change":"2024-04-02T11:32:04.439717+0000","last_active":"2024-04-02T11:33:44.479267+0000","last_peered":"2024-04-02T11:33:44.479267+0000","last_clean":"2024-04-02T11:33:44.479267+0000","last_became_active":"2024-04-02T11:32:04.424122+0000","last_became_peered":"2024-04-02T11:32:04.424122+0000","last_unstale":"2024-04-02T11:33:44.479267+0000","last_undegraded":"2024-04-02T11:33:44.479267+0000","last_fullsized":"2024-04-02T11:33:44.479267+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.0","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.109001+0000","last_change":"2024-04-02T11:33:41.305416+0000","last_active":"2024-04-02T11:33:44.109001+0000","last_peered":"2024-04-02T11:33:44.109001+0000","last_clean":"2024-04-02T11:33:44.109001+0000","last_became_active":"2024-04-02T11:33:41.268384+0000","last_became_peered":"2024-04-02T11:33:41.268384+0000","last_unstale":"2024-04-02T11:33:44.109001+0000","last_undegraded":"2024-04-02T11:33:44.109001+0000","last_fullsized":"2024-04-02T11:33:44.109001+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.608850+0000","last_change":"2024-04-02T11:32:04.410624+0000","last_active":"2024-04-02T11:33:44.608850+0000","last_peered":"2024-04-02T11:33:44.608850+0000","last_clean":"2024-04-02T11:33:44.608850+0000","last_became_active":"2024-04-02T11:32:04.394162+0000","last_became_peered":"2024-04-02T11:32:04.394162+0000","last_unstale":"2024-04-02T11:33:44.608850+0000","last_undegraded":"2024-04-02T11:33:44.608850+0000","last_fullsized":"2024-04-02T11:33:44.608850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.1","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773986+0000","last_change":"2024-04-02T11:33:41.330641+0000","last_active":"2024-04-02T11:33:44.773986+0000","last_peered":"2024-04-02T11:33:44.773986+0000","last_clean":"2024-04-02T11:33:44.773986+0000","last_became_active":"2024-04-02T11:33:41.317736+0000","last_became_peered":"2024-04-02T11:33:41.317736+0000","last_unstale":"2024-04-02T11:33:44.773986+0000","last_undegraded":"2024-04-02T11:33:44.773986+0000","last_fullsized":"2024-04-02T11:33:44.773986+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.771453+0000","last_change":"2024-04-02T11:32:04.455109+0000","last_active":"2024-04-02T11:33:44.771453+0000","last_peered":"2024-04-02T11:33:44.771453+0000","last_clean":"2024-04-02T11:33:44.771453+0000","last_became_active":"2024-04-02T11:32:04.430454+0000","last_became_peered":"2024-04-02T11:32:04.430454+0000","last_unstale":"2024-04-02T11:33:44.771453+0000","last_undegraded":"2024-04-02T11:33:44.771453+0000","last_fullsized":"2024-04-02T11:33:44.771453+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.6","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.574697+0000","last_change":"2024-04-02T11:33:41.252370+0000","last_active":"2024-04-02T11:33:44.574697+0000","last_peered":"2024-04-02T11:33:44.574697+0000","last_clean":"2024-04-02T11:33:44.574697+0000","last_became_active":"2024-04-02T11:33:41.236833+0000","last_became_peered":"2024-04-02T11:33:41.236833+0000","last_unstale":"2024-04-02T11:33:44.574697+0000","last_undegraded":"2024-04-02T11:33:44.574697+0000","last_fullsized":"2024-04-02T11:33:44.574697+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.115811+0000","last_change":"2024-04-02T11:32:04.397555+0000","last_active":"2024-04-02T11:33:44.115811+0000","last_peered":"2024-04-02T11:33:44.115811+0000","last_clean":"2024-04-02T11:33:44.115811+0000","last_became_active":"2024-04-02T11:32:04.392132+0000","last_became_peered":"2024-04-02T11:32:04.392132+0000","last_unstale":"2024-04-02T11:33:44.115811+0000","last_undegraded":"2024-04-02T11:33:44.115811+0000","last_fullsized":"2024-04-02T11:33:44.115811+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.7","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.708501+0000","last_change":"2024-04-02T11:33:41.301764+0000","last_active":"2024-04-02T11:33:44.708501+0000","last_peered":"2024-04-02T11:33:44.708501+0000","last_clean":"2024-04-02T11:33:44.708501+0000","last_became_active":"2024-04-02T11:33:41.288766+0000","last_became_peered":"2024-04-02T11:33:41.288766+0000","last_unstale":"2024-04-02T11:33:44.708501+0000","last_undegraded":"2024-04-02T11:33:44.708501+0000","last_fullsized":"2024-04-02T11:33:44.708501+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.464372+0000","last_change":"2024-04-02T11:32:04.448308+0000","last_active":"2024-04-02T11:33:44.464372+0000","last_peered":"2024-04-02T11:33:44.464372+0000","last_clean":"2024-04-02T11:33:44.464372+0000","last_became_active":"2024-04-02T11:32:04.435642+0000","last_became_peered":"2024-04-02T11:32:04.435642+0000","last_unstale":"2024-04-02T11:33:44.464372+0000","last_undegraded":"2024-04-02T11:33:44.464372+0000","last_fullsized":"2024-04-02T11:33:44.464372+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.4","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.746493+0000","last_change":"2024-04-02T11:33:41.237088+0000","last_active":"2024-04-02T11:33:44.746493+0000","last_peered":"2024-04-02T11:33:44.746493+0000","last_clean":"2024-04-02T11:33:44.746493+0000","last_became_active":"2024-04-02T11:33:41.230755+0000","last_became_peered":"2024-04-02T11:33:41.230755+0000","last_unstale":"2024-04-02T11:33:44.746493+0000","last_undegraded":"2024-04-02T11:33:44.746493+0000","last_fullsized":"2024-04-02T11:33:44.746493+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.476796+0000","last_change":"2024-04-02T11:32:04.531627+0000","last_active":"2024-04-02T11:33:44.476796+0000","last_peered":"2024-04-02T11:33:44.476796+0000","last_clean":"2024-04-02T11:33:44.476796+0000","last_became_active":"2024-04-02T11:32:04.523227+0000","last_became_peered":"2024-04-02T11:32:04.523227+0000","last_unstale":"2024-04-02T11:33:44.476796+0000","last_undegraded":"2024-04-02T11:33:44.476796+0000","last_fullsized":"2024-04-02T11:33:44.476796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.5","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.613522+0000","last_change":"2024-04-02T11:33:41.243595+0000","last_active":"2024-04-02T11:33:44.613522+0000","last_peered":"2024-04-02T11:33:44.613522+0000","last_clean":"2024-04-02T11:33:44.613522+0000","last_became_active":"2024-04-02T11:33:41.229522+0000","last_became_peered":"2024-04-02T11:33:41.229522+0000","last_unstale":"2024-04-02T11:33:44.613522+0000","last_undegraded":"2024-04-02T11:33:44.613522+0000","last_fullsized":"2024-04-02T11:33:44.613522+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.508048+0000","last_change":"2024-04-02T11:32:04.499203+0000","last_active":"2024-04-02T11:33:44.508048+0000","last_peered":"2024-04-02T11:33:44.508048+0000","last_clean":"2024-04-02T11:33:44.508048+0000","last_became_active":"2024-04-02T11:32:04.490360+0000","last_became_peered":"2024-04-02T11:32:04.490360+0000","last_unstale":"2024-04-02T11:33:44.508048+0000","last_undegraded":"2024-04-02T11:33:44.508048+0000","last_fullsized":"2024-04-02T11:33:44.508048+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.619749+0000","last_change":"2024-04-02T11:33:41.308637+0000","last_active":"2024-04-02T11:33:44.619749+0000","last_peered":"2024-04-02T11:33:44.619749+0000","last_clean":"2024-04-02T11:33:44.619749+0000","last_became_active":"2024-04-02T11:33:41.301597+0000","last_became_peered":"2024-04-02T11:33:41.301597+0000","last_unstale":"2024-04-02T11:33:44.619749+0000","last_undegraded":"2024-04-02T11:33:44.619749+0000","last_fullsized":"2024-04-02T11:33:44.619749+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.751562+0000","last_change":"2024-04-02T11:32:04.460524+0000","last_active":"2024-04-02T11:33:44.751562+0000","last_peered":"2024-04-02T11:33:44.751562+0000","last_clean":"2024-04-02T11:33:44.751562+0000","last_became_active":"2024-04-02T11:32:04.427391+0000","last_became_peered":"2024-04-02T11:32:04.427391+0000","last_unstale":"2024-04-02T11:33:44.751562+0000","last_undegraded":"2024-04-02T11:33:44.751562+0000","last_fullsized":"2024-04-02T11:33:44.751562+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.087235+0000","last_change":"2024-04-02T11:33:41.282206+0000","last_active":"2024-04-02T11:33:44.087235+0000","last_peered":"2024-04-02T11:33:44.087235+0000","last_clean":"2024-04-02T11:33:44.087235+0000","last_became_active":"2024-04-02T11:33:41.264133+0000","last_became_peered":"2024-04-02T11:33:41.264133+0000","last_unstale":"2024-04-02T11:33:44.087235+0000","last_undegraded":"2024-04-02T11:33:44.087235+0000","last_fullsized":"2024-04-02T11:33:44.087235+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.618701+0000","last_change":"2024-04-02T11:32:04.403716+0000","last_active":"2024-04-02T11:33:44.618701+0000","last_peered":"2024-04-02T11:33:44.618701+0000","last_clean":"2024-04-02T11:33:44.618701+0000","last_became_active":"2024-04-02T11:32:04.390779+0000","last_became_peered":"2024-04-02T11:32:04.390779+0000","last_unstale":"2024-04-02T11:33:44.618701+0000","last_undegraded":"2024-04-02T11:33:44.618701+0000","last_fullsized":"2024-04-02T11:33:44.618701+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.8","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.743680+0000","last_change":"2024-04-02T11:33:41.322754+0000","last_active":"2024-04-02T11:33:44.743680+0000","last_peered":"2024-04-02T11:33:44.743680+0000","last_clean":"2024-04-02T11:33:44.743680+0000","last_became_active":"2024-04-02T11:33:41.311852+0000","last_became_peered":"2024-04-02T11:33:41.311852+0000","last_unstale":"2024-04-02T11:33:44.743680+0000","last_undegraded":"2024-04-02T11:33:44.743680+0000","last_fullsized":"2024-04-02T11:33:44.743680+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.749745+0000","last_change":"2024-04-02T11:32:04.441727+0000","last_active":"2024-04-02T11:33:44.749745+0000","last_peered":"2024-04-02T11:33:44.749745+0000","last_clean":"2024-04-02T11:33:44.749745+0000","last_became_active":"2024-04-02T11:32:04.409646+0000","last_became_peered":"2024-04-02T11:32:04.409646+0000","last_unstale":"2024-04-02T11:33:44.749745+0000","last_undegraded":"2024-04-02T11:33:44.749745+0000","last_fullsized":"2024-04-02T11:33:44.749745+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.9","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773128+0000","last_change":"2024-04-02T11:33:41.293468+0000","last_active":"2024-04-02T11:33:44.773128+0000","last_peered":"2024-04-02T11:33:44.773128+0000","last_clean":"2024-04-02T11:33:44.773128+0000","last_became_active":"2024-04-02T11:33:41.267676+0000","last_became_peered":"2024-04-02T11:33:41.267676+0000","last_unstale":"2024-04-02T11:33:44.773128+0000","last_undegraded":"2024-04-02T11:33:44.773128+0000","last_fullsized":"2024-04-02T11:33:44.773128+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.509804+0000","last_change":"2024-04-02T11:32:04.404984+0000","last_active":"2024-04-02T11:33:44.509804+0000","last_peered":"2024-04-02T11:33:44.509804+0000","last_clean":"2024-04-02T11:33:44.509804+0000","last_became_active":"2024-04-02T11:32:04.387697+0000","last_became_peered":"2024-04-02T11:32:04.387697+0000","last_unstale":"2024-04-02T11:33:44.509804+0000","last_undegraded":"2024-04-02T11:33:44.509804+0000","last_fullsized":"2024-04-02T11:33:44.509804+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.755181+0000","last_change":"2024-04-02T11:33:41.296184+0000","last_active":"2024-04-02T11:33:44.755181+0000","last_peered":"2024-04-02T11:33:44.755181+0000","last_clean":"2024-04-02T11:33:44.755181+0000","last_became_active":"2024-04-02T11:33:41.271175+0000","last_became_peered":"2024-04-02T11:33:41.271175+0000","last_unstale":"2024-04-02T11:33:44.755181+0000","last_undegraded":"2024-04-02T11:33:44.755181+0000","last_fullsized":"2024-04-02T11:33:44.755181+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.732609+0000","last_change":"2024-04-02T11:32:04.423708+0000","last_active":"2024-04-02T11:33:44.732609+0000","last_peered":"2024-04-02T11:33:44.732609+0000","last_clean":"2024-04-02T11:33:44.732609+0000","last_became_active":"2024-04-02T11:32:04.401780+0000","last_became_peered":"2024-04-02T11:32:04.401780+0000","last_unstale":"2024-04-02T11:33:44.732609+0000","last_undegraded":"2024-04-02T11:33:44.732609+0000","last_fullsized":"2024-04-02T11:33:44.732609+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.742027+0000","last_change":"2024-04-02T11:33:41.274959+0000","last_active":"2024-04-02T11:33:44.742027+0000","last_peered":"2024-04-02T11:33:44.742027+0000","last_clean":"2024-04-02T11:33:44.742027+0000","last_became_active":"2024-04-02T11:33:41.254717+0000","last_became_peered":"2024-04-02T11:33:41.254717+0000","last_unstale":"2024-04-02T11:33:44.742027+0000","last_undegraded":"2024-04-02T11:33:44.742027+0000","last_fullsized":"2024-04-02T11:33:44.742027+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.728056+0000","last_change":"2024-04-02T11:32:04.457456+0000","last_active":"2024-04-02T11:33:44.728056+0000","last_peered":"2024-04-02T11:33:44.728056+0000","last_clean":"2024-04-02T11:33:44.728056+0000","last_became_active":"2024-04-02T11:32:04.436936+0000","last_became_peered":"2024-04-02T11:32:04.436936+0000","last_unstale":"2024-04-02T11:33:44.728056+0000","last_undegraded":"2024-04-02T11:33:44.728056+0000","last_fullsized":"2024-04-02T11:33:44.728056+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.579024+0000","last_change":"2024-04-02T11:33:41.259181+0000","last_active":"2024-04-02T11:33:44.579024+0000","last_peered":"2024-04-02T11:33:44.579024+0000","last_clean":"2024-04-02T11:33:44.579024+0000","last_became_active":"2024-04-02T11:33:41.249228+0000","last_became_peered":"2024-04-02T11:33:41.249228+0000","last_unstale":"2024-04-02T11:33:44.579024+0000","last_undegraded":"2024-04-02T11:33:44.579024+0000","last_fullsized":"2024-04-02T11:33:44.579024+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.503670+0000","last_change":"2024-04-02T11:32:04.429226+0000","last_active":"2024-04-02T11:33:44.503670+0000","last_peered":"2024-04-02T11:33:44.503670+0000","last_clean":"2024-04-02T11:33:44.503670+0000","last_became_active":"2024-04-02T11:32:04.407776+0000","last_became_peered":"2024-04-02T11:32:04.407776+0000","last_unstale":"2024-04-02T11:33:44.503670+0000","last_undegraded":"2024-04-02T11:33:44.503670+0000","last_fullsized":"2024-04-02T11:33:44.503670+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.480198+0000","last_change":"2024-04-02T11:33:41.329049+0000","last_active":"2024-04-02T11:33:44.480198+0000","last_peered":"2024-04-02T11:33:44.480198+0000","last_clean":"2024-04-02T11:33:44.480198+0000","last_became_active":"2024-04-02T11:33:41.323281+0000","last_became_peered":"2024-04-02T11:33:41.323281+0000","last_unstale":"2024-04-02T11:33:44.480198+0000","last_undegraded":"2024-04-02T11:33:44.480198+0000","last_fullsized":"2024-04-02T11:33:44.480198+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.494895+0000","last_change":"2024-04-02T11:32:04.558136+0000","last_active":"2024-04-02T11:33:44.494895+0000","last_peered":"2024-04-02T11:33:44.494895+0000","last_clean":"2024-04-02T11:33:44.494895+0000","last_became_active":"2024-04-02T11:32:04.551595+0000","last_became_peered":"2024-04-02T11:32:04.551595+0000","last_unstale":"2024-04-02T11:33:44.494895+0000","last_undegraded":"2024-04-02T11:33:44.494895+0000","last_fullsized":"2024-04-02T11:33:44.494895+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.12","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.478405+0000","last_change":"2024-04-02T11:33:41.281874+0000","last_active":"2024-04-02T11:33:44.478405+0000","last_peered":"2024-04-02T11:33:44.478405+0000","last_clean":"2024-04-02T11:33:44.478405+0000","last_became_active":"2024-04-02T11:33:41.274203+0000","last_became_peered":"2024-04-02T11:33:41.274203+0000","last_unstale":"2024-04-02T11:33:44.478405+0000","last_undegraded":"2024-04-02T11:33:44.478405+0000","last_fullsized":"2024-04-02T11:33:44.478405+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.485930+0000","last_change":"2024-04-02T11:32:04.417231+0000","last_active":"2024-04-02T11:33:44.485930+0000","last_peered":"2024-04-02T11:33:44.485930+0000","last_clean":"2024-04-02T11:33:44.485930+0000","last_became_active":"2024-04-02T11:32:04.396181+0000","last_became_peered":"2024-04-02T11:32:04.396181+0000","last_unstale":"2024-04-02T11:33:44.485930+0000","last_undegraded":"2024-04-02T11:33:44.485930+0000","last_fullsized":"2024-04-02T11:33:44.485930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.13","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.602064+0000","last_change":"2024-04-02T11:33:41.245713+0000","last_active":"2024-04-02T11:33:44.602064+0000","last_peered":"2024-04-02T11:33:44.602064+0000","last_clean":"2024-04-02T11:33:44.602064+0000","last_became_active":"2024-04-02T11:33:41.226115+0000","last_became_peered":"2024-04-02T11:33:41.226115+0000","last_unstale":"2024-04-02T11:33:44.602064+0000","last_undegraded":"2024-04-02T11:33:44.602064+0000","last_fullsized":"2024-04-02T11:33:44.602064+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.754146+0000","last_change":"2024-04-02T11:32:04.450092+0000","last_active":"2024-04-02T11:33:44.754146+0000","last_peered":"2024-04-02T11:33:44.754146+0000","last_clean":"2024-04-02T11:33:44.754146+0000","last_became_active":"2024-04-02T11:32:04.418023+0000","last_became_peered":"2024-04-02T11:32:04.418023+0000","last_unstale":"2024-04-02T11:33:44.754146+0000","last_undegraded":"2024-04-02T11:33:44.754146+0000","last_fullsized":"2024-04-02T11:33:44.754146+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.10","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.572668+0000","last_change":"2024-04-02T11:33:41.266477+0000","last_active":"2024-04-02T11:33:44.572668+0000","last_peered":"2024-04-02T11:33:44.572668+0000","last_clean":"2024-04-02T11:33:44.572668+0000","last_became_active":"2024-04-02T11:33:41.256091+0000","last_became_peered":"2024-04-02T11:33:41.256091+0000","last_unstale":"2024-04-02T11:33:44.572668+0000","last_undegraded":"2024-04-02T11:33:44.572668+0000","last_fullsized":"2024-04-02T11:33:44.572668+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.611376+0000","last_change":"2024-04-02T11:32:04.397771+0000","last_active":"2024-04-02T11:33:44.611376+0000","last_peered":"2024-04-02T11:33:44.611376+0000","last_clean":"2024-04-02T11:33:44.611376+0000","last_became_active":"2024-04-02T11:32:04.378859+0000","last_became_peered":"2024-04-02T11:32:04.378859+0000","last_unstale":"2024-04-02T11:33:44.611376+0000","last_undegraded":"2024-04-02T11:33:44.611376+0000","last_fullsized":"2024-04-02T11:33:44.611376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.11","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.758547+0000","last_change":"2024-04-02T11:33:41.263642+0000","last_active":"2024-04-02T11:33:44.758547+0000","last_peered":"2024-04-02T11:33:44.758547+0000","last_clean":"2024-04-02T11:33:44.758547+0000","last_became_active":"2024-04-02T11:33:41.250607+0000","last_became_peered":"2024-04-02T11:33:41.250607+0000","last_unstale":"2024-04-02T11:33:44.758547+0000","last_undegraded":"2024-04-02T11:33:44.758547+0000","last_fullsized":"2024-04-02T11:33:44.758547+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.073706+0000","last_change":"2024-04-02T11:32:04.463785+0000","last_active":"2024-04-02T11:33:44.073706+0000","last_peered":"2024-04-02T11:33:44.073706+0000","last_clean":"2024-04-02T11:33:44.073706+0000","last_became_active":"2024-04-02T11:32:04.457870+0000","last_became_peered":"2024-04-02T11:32:04.457870+0000","last_unstale":"2024-04-02T11:33:44.073706+0000","last_undegraded":"2024-04-02T11:33:44.073706+0000","last_fullsized":"2024-04-02T11:33:44.073706+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.16","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.605469+0000","last_change":"2024-04-02T11:33:41.241536+0000","last_active":"2024-04-02T11:33:44.605469+0000","last_peered":"2024-04-02T11:33:44.605469+0000","last_clean":"2024-04-02T11:33:44.605469+0000","last_became_active":"2024-04-02T11:33:41.232699+0000","last_became_peered":"2024-04-02T11:33:41.232699+0000","last_unstale":"2024-04-02T11:33:44.605469+0000","last_undegraded":"2024-04-02T11:33:44.605469+0000","last_fullsized":"2024-04-02T11:33:44.605469+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.071157+0000","last_change":"2024-04-02T11:32:04.384861+0000","last_active":"2024-04-02T11:33:44.071157+0000","last_peered":"2024-04-02T11:33:44.071157+0000","last_clean":"2024-04-02T11:33:44.071157+0000","last_became_active":"2024-04-02T11:32:04.368304+0000","last_became_peered":"2024-04-02T11:32:04.368304+0000","last_unstale":"2024-04-02T11:33:44.071157+0000","last_undegraded":"2024-04-02T11:33:44.071157+0000","last_fullsized":"2024-04-02T11:33:44.071157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.17","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.590092+0000","last_change":"2024-04-02T11:33:41.281940+0000","last_active":"2024-04-02T11:33:44.590092+0000","last_peered":"2024-04-02T11:33:44.590092+0000","last_clean":"2024-04-02T11:33:44.590092+0000","last_became_active":"2024-04-02T11:33:41.275074+0000","last_became_peered":"2024-04-02T11:33:41.275074+0000","last_unstale":"2024-04-02T11:33:44.590092+0000","last_undegraded":"2024-04-02T11:33:44.590092+0000","last_fullsized":"2024-04-02T11:33:44.590092+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.091094+0000","last_change":"2024-04-02T11:32:04.377416+0000","last_active":"2024-04-02T11:33:44.091094+0000","last_peered":"2024-04-02T11:33:44.091094+0000","last_clean":"2024-04-02T11:33:44.091094+0000","last_became_active":"2024-04-02T11:32:04.364464+0000","last_became_peered":"2024-04-02T11:32:04.364464+0000","last_unstale":"2024-04-02T11:33:44.091094+0000","last_undegraded":"2024-04-02T11:33:44.091094+0000","last_fullsized":"2024-04-02T11:33:44.091094+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.14","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.076833+0000","last_change":"2024-04-02T11:33:41.311816+0000","last_active":"2024-04-02T11:33:44.076833+0000","last_peered":"2024-04-02T11:33:44.076833+0000","last_clean":"2024-04-02T11:33:44.076833+0000","last_became_active":"2024-04-02T11:33:41.290381+0000","last_became_peered":"2024-04-02T11:33:41.290381+0000","last_unstale":"2024-04-02T11:33:44.076833+0000","last_undegraded":"2024-04-02T11:33:44.076833+0000","last_fullsized":"2024-04-02T11:33:44.076833+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.599260+0000","last_change":"2024-04-02T11:32:04.407271+0000","last_active":"2024-04-02T11:33:44.599260+0000","last_peered":"2024-04-02T11:33:44.599260+0000","last_clean":"2024-04-02T11:33:44.599260+0000","last_became_active":"2024-04-02T11:32:04.386636+0000","last_became_peered":"2024-04-02T11:32:04.386636+0000","last_unstale":"2024-04-02T11:33:44.599260+0000","last_undegraded":"2024-04-02T11:33:44.599260+0000","last_fullsized":"2024-04-02T11:33:44.599260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.15","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.085724+0000","last_change":"2024-04-02T11:33:41.299177+0000","last_active":"2024-04-02T11:33:44.085724+0000","last_peered":"2024-04-02T11:33:44.085724+0000","last_clean":"2024-04-02T11:33:44.085724+0000","last_became_active":"2024-04-02T11:33:41.271590+0000","last_became_peered":"2024-04-02T11:33:41.271590+0000","last_unstale":"2024-04-02T11:33:44.085724+0000","last_undegraded":"2024-04-02T11:33:44.085724+0000","last_fullsized":"2024-04-02T11:33:44.085724+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.703445+0000","last_change":"2024-04-02T11:32:04.421594+0000","last_active":"2024-04-02T11:33:44.703445+0000","last_peered":"2024-04-02T11:33:44.703445+0000","last_clean":"2024-04-02T11:33:44.703445+0000","last_became_active":"2024-04-02T11:32:04.404970+0000","last_became_peered":"2024-04-02T11:32:04.404970+0000","last_unstale":"2024-04-02T11:33:44.703445+0000","last_undegraded":"2024-04-02T11:33:44.703445+0000","last_fullsized":"2024-04-02T11:33:44.703445+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113166+0000","last_change":"2024-04-02T11:33:41.294167+0000","last_active":"2024-04-02T11:33:44.113166+0000","last_peered":"2024-04-02T11:33:44.113166+0000","last_clean":"2024-04-02T11:33:44.113166+0000","last_became_active":"2024-04-02T11:33:41.278749+0000","last_became_peered":"2024-04-02T11:33:41.278749+0000","last_unstale":"2024-04-02T11:33:44.113166+0000","last_undegraded":"2024-04-02T11:33:44.113166+0000","last_fullsized":"2024-04-02T11:33:44.113166+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.753274+0000","last_change":"2024-04-02T11:32:04.439591+0000","last_active":"2024-04-02T11:33:44.753274+0000","last_peered":"2024-04-02T11:33:44.753274+0000","last_clean":"2024-04-02T11:33:44.753274+0000","last_became_active":"2024-04-02T11:32:04.414684+0000","last_became_peered":"2024-04-02T11:32:04.414684+0000","last_unstale":"2024-04-02T11:33:44.753274+0000","last_undegraded":"2024-04-02T11:33:44.753274+0000","last_fullsized":"2024-04-02T11:33:44.753274+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.739455+0000","last_change":"2024-04-02T11:33:41.277360+0000","last_active":"2024-04-02T11:33:44.739455+0000","last_peered":"2024-04-02T11:33:44.739455+0000","last_clean":"2024-04-02T11:33:44.739455+0000","last_became_active":"2024-04-02T11:33:41.258789+0000","last_became_peered":"2024-04-02T11:33:41.258789+0000","last_unstale":"2024-04-02T11:33:44.739455+0000","last_undegraded":"2024-04-02T11:33:44.739455+0000","last_fullsized":"2024-04-02T11:33:44.739455+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.510847+0000","last_change":"2024-04-02T11:32:04.419077+0000","last_active":"2024-04-02T11:33:44.510847+0000","last_peered":"2024-04-02T11:33:44.510847+0000","last_clean":"2024-04-02T11:33:44.510847+0000","last_became_active":"2024-04-02T11:32:04.392023+0000","last_became_peered":"2024-04-02T11:32:04.392023+0000","last_unstale":"2024-04-02T11:33:44.510847+0000","last_undegraded":"2024-04-02T11:33:44.510847+0000","last_fullsized":"2024-04-02T11:33:44.510847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.18","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.501017+0000","last_change":"2024-04-02T11:33:41.379485+0000","last_active":"2024-04-02T11:33:44.501017+0000","last_peered":"2024-04-02T11:33:44.501017+0000","last_clean":"2024-04-02T11:33:44.501017+0000","last_became_active":"2024-04-02T11:33:41.374309+0000","last_became_peered":"2024-04-02T11:33:41.374309+0000","last_unstale":"2024-04-02T11:33:44.501017+0000","last_undegraded":"2024-04-02T11:33:44.501017+0000","last_fullsized":"2024-04-02T11:33:44.501017+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.766807+0000","last_change":"2024-04-02T11:32:04.464793+0000","last_active":"2024-04-02T11:33:44.766807+0000","last_peered":"2024-04-02T11:33:44.766807+0000","last_clean":"2024-04-02T11:33:44.766807+0000","last_became_active":"2024-04-02T11:32:04.445170+0000","last_became_peered":"2024-04-02T11:32:04.445170+0000","last_unstale":"2024-04-02T11:33:44.766807+0000","last_undegraded":"2024-04-02T11:33:44.766807+0000","last_fullsized":"2024-04-02T11:33:44.766807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.19","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.493963+0000","last_change":"2024-04-02T11:33:41.353471+0000","last_active":"2024-04-02T11:33:44.493963+0000","last_peered":"2024-04-02T11:33:44.493963+0000","last_clean":"2024-04-02T11:33:44.493963+0000","last_became_active":"2024-04-02T11:33:41.347559+0000","last_became_peered":"2024-04-02T11:33:41.347559+0000","last_unstale":"2024-04-02T11:33:44.493963+0000","last_undegraded":"2024-04-02T11:33:44.493963+0000","last_fullsized":"2024-04-02T11:33:44.493963+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.492318+0000","last_change":"2024-04-02T11:32:04.474632+0000","last_active":"2024-04-02T11:33:44.492318+0000","last_peered":"2024-04-02T11:33:44.492318+0000","last_clean":"2024-04-02T11:33:44.492318+0000","last_became_active":"2024-04-02T11:32:04.467593+0000","last_became_peered":"2024-04-02T11:32:04.467593+0000","last_unstale":"2024-04-02T11:33:44.492318+0000","last_undegraded":"2024-04-02T11:33:44.492318+0000","last_fullsized":"2024-04-02T11:33:44.492318+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.765959+0000","last_change":"2024-04-02T11:33:41.303867+0000","last_active":"2024-04-02T11:33:44.765959+0000","last_peered":"2024-04-02T11:33:44.765959+0000","last_clean":"2024-04-02T11:33:44.765959+0000","last_became_active":"2024-04-02T11:33:41.282720+0000","last_became_peered":"2024-04-02T11:33:41.282720+0000","last_unstale":"2024-04-02T11:33:44.765959+0000","last_undegraded":"2024-04-02T11:33:44.765959+0000","last_fullsized":"2024-04-02T11:33:44.765959+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.489009+0000","last_change":"2024-04-02T11:32:04.415345+0000","last_active":"2024-04-02T11:33:44.489009+0000","last_peered":"2024-04-02T11:33:44.489009+0000","last_clean":"2024-04-02T11:33:44.489009+0000","last_became_active":"2024-04-02T11:32:04.400375+0000","last_became_peered":"2024-04-02T11:32:04.400375+0000","last_unstale":"2024-04-02T11:33:44.489009+0000","last_undegraded":"2024-04-02T11:33:44.489009+0000","last_fullsized":"2024-04-02T11:33:44.489009+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.777113+0000","last_change":"2024-04-02T11:33:41.334657+0000","last_active":"2024-04-02T11:33:44.777113+0000","last_peered":"2024-04-02T11:33:44.777113+0000","last_clean":"2024-04-02T11:33:44.777113+0000","last_became_active":"2024-04-02T11:33:41.325633+0000","last_became_peered":"2024-04-02T11:33:41.325633+0000","last_unstale":"2024-04-02T11:33:44.777113+0000","last_undegraded":"2024-04-02T11:33:44.777113+0000","last_fullsized":"2024-04-02T11:33:44.777113+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.471747+0000","last_change":"2024-04-02T11:32:04.461967+0000","last_active":"2024-04-02T11:33:44.471747+0000","last_peered":"2024-04-02T11:33:44.471747+0000","last_clean":"2024-04-02T11:33:44.471747+0000","last_became_active":"2024-04-02T11:32:04.456013+0000","last_became_peered":"2024-04-02T11:32:04.456013+0000","last_unstale":"2024-04-02T11:33:44.471747+0000","last_undegraded":"2024-04-02T11:33:44.471747+0000","last_fullsized":"2024-04-02T11:33:44.471747+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.506496+0000","last_change":"2024-04-02T11:33:41.285447+0000","last_active":"2024-04-02T11:33:44.506496+0000","last_peered":"2024-04-02T11:33:44.506496+0000","last_clean":"2024-04-02T11:33:44.506496+0000","last_became_active":"2024-04-02T11:33:41.270749+0000","last_became_peered":"2024-04-02T11:33:41.270749+0000","last_unstale":"2024-04-02T11:33:44.506496+0000","last_undegraded":"2024-04-02T11:33:44.506496+0000","last_fullsized":"2024-04-02T11:33:44.506496+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":151,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.074943+0000","last_change":"2024-04-02T11:32:04.431094+0000","last_active":"2024-04-02T11:33:44.074943+0000","last_peered":"2024-04-02T11:33:44.074943+0000","last_clean":"2024-04-02T11:33:44.074943+0000","last_became_active":"2024-04-02T11:32:04.423058+0000","last_became_peered":"2024-04-02T11:32:04.423058+0000","last_unstale":"2024-04-02T11:33:44.074943+0000","last_undegraded":"2024-04-02T11:33:44.074943+0000","last_fullsized":"2024-04-02T11:33:44.074943+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.1d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.106087+0000","last_change":"2024-04-02T11:33:41.274565+0000","last_active":"2024-04-02T11:33:44.106087+0000","last_peered":"2024-04-02T11:33:44.106087+0000","last_clean":"2024-04-02T11:33:44.106087+0000","last_became_active":"2024-04-02T11:33:41.258583+0000","last_became_peered":"2024-04-02T11:33:41.258583+0000","last_unstale":"2024-04-02T11:33:44.106087+0000","last_undegraded":"2024-04-02T11:33:44.106087+0000","last_fullsized":"2024-04-02T11:33:44.106087+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.593632+0000","last_change":"2024-04-02T11:32:04.401482+0000","last_active":"2024-04-02T11:33:44.593632+0000","last_peered":"2024-04-02T11:33:44.593632+0000","last_clean":"2024-04-02T11:33:44.593632+0000","last_became_active":"2024-04-02T11:32:04.383344+0000","last_became_peered":"2024-04-02T11:32:04.383344+0000","last_unstale":"2024-04-02T11:33:44.593632+0000","last_undegraded":"2024-04-02T11:33:44.593632+0000","last_fullsized":"2024-04-02T11:33:44.593632+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.069239+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:33:44.069239+0000","last_peered":"2024-04-02T11:33:44.069239+0000","last_clean":"2024-04-02T11:33:44.069239+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:33:44.069239+0000","last_undegraded":"2024-04-02T11:33:44.069239+0000","last_fullsized":"2024-04-02T11:33:44.069239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.627702+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:33:44.627702+0000","last_peered":"2024-04-02T11:33:44.627702+0000","last_clean":"2024-04-02T11:33:44.627702+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:33:44.627702+0000","last_undegraded":"2024-04-02T11:33:44.627702+0000","last_fullsized":"2024-04-02T11:33:44.627702+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":741,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.722011+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:33:44.722011+0000","last_peered":"2024-04-02T11:33:44.722011+0000","last_clean":"2024-04-02T11:33:44.722011+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:33:44.722011+0000","last_undegraded":"2024-04-02T11:33:44.722011+0000","last_fullsized":"2024-04-02T11:33:44.722011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.623600+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:33:44.623600+0000","last_peered":"2024-04-02T11:33:44.623600+0000","last_clean":"2024-04-02T11:33:44.623600+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:33:44.623600+0000","last_undegraded":"2024-04-02T11:33:44.623600+0000","last_fullsized":"2024-04-02T11:33:44.623600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.473736+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:33:44.473736+0000","last_peered":"2024-04-02T11:33:44.473736+0000","last_clean":"2024-04-02T11:33:44.473736+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:33:44.473736+0000","last_undegraded":"2024-04-02T11:33:44.473736+0000","last_fullsized":"2024-04-02T11:33:44.473736+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.110855+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:33:44.110855+0000","last_peered":"2024-04-02T11:33:44.110855+0000","last_clean":"2024-04-02T11:33:44.110855+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:33:44.110855+0000","last_undegraded":"2024-04-02T11:33:44.110855+0000","last_fullsized":"2024-04-02T11:33:44.110855+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.504908+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:33:44.504908+0000","last_peered":"2024-04-02T11:33:44.504908+0000","last_clean":"2024-04-02T11:33:44.504908+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:33:44.504908+0000","last_undegraded":"2024-04-02T11:33:44.504908+0000","last_fullsized":"2024-04-02T11:33:44.504908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113298+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:33:44.113298+0000","last_peered":"2024-04-02T11:33:44.113298+0000","last_clean":"2024-04-02T11:33:44.113298+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:33:44.113298+0000","last_undegraded":"2024-04-02T11:33:44.113298+0000","last_fullsized":"2024-04-02T11:33:44.113298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":877,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.734120+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:33:44.734120+0000","last_peered":"2024-04-02T11:33:44.734120+0000","last_clean":"2024-04-02T11:33:44.734120+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:33:44.734120+0000","last_undegraded":"2024-04-02T11:33:44.734120+0000","last_fullsized":"2024-04-02T11:33:44.734120+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":86,"num_pg":32,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":4},{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":31104,"kb_avail":104824672,"statfs":{"total":107374182400,"available":107340464128,"internally_reserved":0,"allocated":1859584,"data_stored":1037844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31850496},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829396,"statfs":{"total":107374182400,"available":107345301504,"internally_reserved":0,"allocated":1282048,"data_stored":451769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654716520,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":54576,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104803024,"statfs":{"total":107374182400,"available":107318296576,"internally_reserved":0,"allocated":1286144,"data_stored":456057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654716519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":55140,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104802460,"statfs":{"total":107374182400,"available":107317719040,"internally_reserved":0,"allocated":1863680,"data_stored":1037972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:28:01.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:28:01.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.275+0000 7fcddd59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:01.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.280+0000 7fcddd59d640 1 Processor -- start 2024-04-02T14:28:01.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.280+0000 7fcddd59d640 1 -- start start 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcddd59d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcdd8131780 0x7fcdd8131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcddd59d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd80a6af0 con 0x7fcdd809edf0 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcddd59d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd8073660 con 0x7fcdd8131780 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcdd8131780 0x7fcdd8131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcdd8131780 0x7fcdd8131b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47756/0 (socket says 172.21.15.133:47756) 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdce7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcdd809edf0 0x7fcdd809f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37004/0 (socket says 172.21.15.133:37004) 2024-04-02T14:28:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 -- 172.21.15.133:0/1996042837 learned_addr learned my addr 172.21.15.133:0/1996042837 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 -- 172.21.15.133:0/1996042837 >> v1:172.21.15.133:6789/0 conn(0x7fcdd809edf0 legacy=0x7fcdd809f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 -- 172.21.15.133:0/1996042837 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd80a67e0 con 0x7fcdd8131780 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd6ffd640 1 --2- 172.21.15.133:0/1996042837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd8131780 0x7fcdd8131b90 crc :-1 s=READY pgs=12317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60766179761ac32a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.281+0000 7fcdd67fc640 1 -- 172.21.15.133:0/1996042837 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdc8009e50 con 0x7fcdd8131780 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcdd67fc640 1 -- 172.21.15.133:0/1996042837 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcdc800e230 con 0x7fcdd8131780 2024-04-02T14:28:01.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcdd67fc640 1 -- 172.21.15.133:0/1996042837 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdc800e4d0 con 0x7fcdd8131780 2024-04-02T14:28:01.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 -- 172.21.15.133:0/1996042837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd8131780 msgr2=0x7fcdd8131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 --2- 172.21.15.133:0/1996042837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd8131780 0x7fcdd8131b90 crc :-1 s=READY pgs=12317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 -- 172.21.15.133:0/1996042837 shutdown_connections 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 --2- 172.21.15.133:0/1996042837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd8131780 0x7fcdd8131b90 unknown :-1 s=CLOSED pgs=12317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 -- 172.21.15.133:0/1996042837 >> 172.21.15.133:0/1996042837 conn(0x7fcdd80a4ba0 msgr2=0x7fcdd80a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 -- 172.21.15.133:0/1996042837 shutdown_connections 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.282+0000 7fcddd59d640 1 -- 172.21.15.133:0/1996042837 wait complete. 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.283+0000 7fcddd59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.284+0000 7fcddd59d640 1 Processor -- start 2024-04-02T14:28:01.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.284+0000 7fcddd59d640 1 -- start start 2024-04-02T14:28:01.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcddd59d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:01.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcddd59d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8121080 con 0x7fcdd809edf0 2024-04-02T14:28:01.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:01.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47762/0 (socket says 172.21.15.133:47762) 2024-04-02T14:28:01.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcdd6ffd640 1 -- 172.21.15.133:0/3077003183 learned_addr learned my addr 172.21.15.133:0/3077003183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.285+0000 7fcdd6ffd640 1 -- 172.21.15.133:0/3077003183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd811e410 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcdd6ffd640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 crc :-1 s=READY pgs=12318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdc800eda0 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcdc8015420 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd811f400 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcdc80156c0 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.286+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd811fc00 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.287+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd8063400 con 0x7fcdd809edf0 2024-04-02T14:28:01.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.287+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fcdc80158c0 con 0x7fcdd809edf0 2024-04-02T14:28:01.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.287+0000 7fcdcffff640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 0x7fcdb803ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:01.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.287+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fcdc8049b60 con 0x7fcdd809edf0 2024-04-02T14:28:01.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.288+0000 7fcdce7fc640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 0x7fcdb803ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:01.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.288+0000 7fcdce7fc640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 0x7fcdb803ae40 crc :-1 s=READY pgs=3801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:01.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.293+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fcdc8010c30 con 0x7fcdd809edf0 2024-04-02T14:28:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.475+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdd811f770 con 0x7fcdd809edf0 2024-04-02T14:28:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.476+0000 7fcdcffff640 1 -- 172.21.15.133:0/3077003183 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v287) ==== 74+0+12453 (crc 0 0 0) 0x7fcdc8010e60 con 0x7fcdd809edf0 2024-04-02T14:28:01.478 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:01.478 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":287,"fsid":"0fac6a5d-b779-400c-9b2f-b14c76190269","created":"2024-04-02T11:26:43.980379+0000","modified":"2024-04-02T11:33:44.042243+0000","last_up_change":"2024-04-02T11:26:50.078833+0000","last_in_change":"2024-04-02T11:26:46.106374+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":86,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T11:26:50.584783+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-02T11:26:54.219118+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-04-02T11:32:02.227788+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":86,"pool_name":"LibRadosChecksum/2_smithi133-95939-30","create_time":"2024-04-02T11:33:39.072880+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"286","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"98a4ef3a-6098-49fd-8bf6-b5d47db5b01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":284,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6801","nonce":1793372331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1793372331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1793372331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":1793372331}]},"public_addr":"172.21.15.133:6801/1793372331","cluster_addr":"172.21.15.133:6802/1793372331","heartbeat_back_addr":"172.21.15.133:6806/1793372331","heartbeat_front_addr":"172.21.15.133:6807/1793372331","state":["exists","up"]},{"osd":1,"uuid":"a7ca277f-c9c9-4571-9fcd-b183ed099431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":284,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":4121831248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4121831248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":4121831248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6813","nonce":4121831248}]},"public_addr":"172.21.15.133:6803/4121831248","cluster_addr":"172.21.15.133:6800/4121831248","heartbeat_back_addr":"172.21.15.133:6814/4121831248","heartbeat_front_addr":"172.21.15.133:6813/4121831248","state":["exists","up"]},{"osd":2,"uuid":"d4688372-a095-4687-9a15-67e86164a07c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":284,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6805","nonce":1859851880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1859851880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":1859851880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1859851880}]},"public_addr":"172.21.15.138:6805/1859851880","cluster_addr":"172.21.15.138:6804/1859851880","heartbeat_back_addr":"172.21.15.138:6811/1859851880","heartbeat_front_addr":"172.21.15.138:6812/1859851880","state":["exists","up"]},{"osd":3,"uuid":"514ade77-a0ef-4940-84f3-2a75ebe04025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":284,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3056026735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":3056026735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3056026735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":3056026735}]},"public_addr":"172.21.15.138:6806/3056026735","cluster_addr":"172.21.15.138:6807/3056026735","heartbeat_back_addr":"172.21.15.138:6808/3056026735","heartbeat_front_addr":"172.21.15.138:6809/3056026735","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:28:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.478+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 msgr2=0x7fcdb803ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.478+0000 7fcddd59d640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 0x7fcdb803ae40 crc :-1 s=READY pgs=3801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.478+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 msgr2=0x7fcdd81201f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.478+0000 7fcddd59d640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 crc :-1 s=READY pgs=12318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 shutdown_connections 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fcdb8038930 0x7fcdb803ae40 unknown :-1 s=CLOSED pgs=3801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 --2- 172.21.15.133:0/3077003183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdd809edf0 0x7fcdd81201f0 unknown :-1 s=CLOSED pgs=12318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 >> 172.21.15.133:0/3077003183 conn(0x7fcdd80a4ba0 msgr2=0x7fcdd8133640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 shutdown_connections 2024-04-02T14:28:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:01.479+0000 7fcddd59d640 1 -- 172.21.15.133:0/3077003183 wait complete. 2024-04-02T14:28:02.493 INFO:tasks.ceph:Scrubbing osd.0 2024-04-02T14:28:02.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-02T14:28:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.604+0000 7f3fca82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.613+0000 7f3fca82d640 1 Processor -- start 2024-04-02T14:28:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.613+0000 7f3fca82d640 1 -- start start 2024-04-02T14:28:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fca82d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3fc40a0d60 0x7f3fc41349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fca82d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc41345d0 con 0x7f3fc4134f00 2024-04-02T14:28:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fca82d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc40a0b40 con 0x7f3fc40a0d60 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3fc40a0d60 0x7f3fc41349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3fc4134f00 0x7f3fc409edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37008/0 (socket says 172.21.15.133:37008) 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc37fe640 1 -- 172.21.15.133:0/3127471362 learned_addr learned my addr 172.21.15.133:0/3127471362 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc3fff640 1 -- 172.21.15.133:0/3127471362 >> v1:172.21.15.133:6789/0 conn(0x7f3fc4134f00 legacy=0x7f3fc409edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc3fff640 1 -- 172.21.15.133:0/3127471362 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc406ea50 con 0x7f3fc40a0d60 2024-04-02T14:28:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc3fff640 1 --2- 172.21.15.133:0/3127471362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc40a0d60 0x7f3fc41349c0 crc :-1 s=READY pgs=12320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6121aadb4e2b202e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc2ffd640 1 -- 172.21.15.133:0/3127471362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3fb8009d90 con 0x7f3fc40a0d60 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.614+0000 7f3fc2ffd640 1 -- 172.21.15.133:0/3127471362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3fb800e230 con 0x7f3fc40a0d60 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fc2ffd640 1 -- 172.21.15.133:0/3127471362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3fb800e4d0 con 0x7f3fc40a0d60 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 -- 172.21.15.133:0/3127471362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc40a0d60 msgr2=0x7f3fc41349c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 --2- 172.21.15.133:0/3127471362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc40a0d60 0x7f3fc41349c0 crc :-1 s=READY pgs=12320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 -- 172.21.15.133:0/3127471362 shutdown_connections 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 --2- 172.21.15.133:0/3127471362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc40a0d60 0x7f3fc41349c0 unknown :-1 s=CLOSED pgs=12320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 -- 172.21.15.133:0/3127471362 >> 172.21.15.133:0/3127471362 conn(0x7f3fc40a4a90 msgr2=0x7f3fc40a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 -- 172.21.15.133:0/3127471362 shutdown_connections 2024-04-02T14:28:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 1 -- 172.21.15.133:0/3127471362 wait complete. 2024-04-02T14:28:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.615+0000 7f3fca82d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:02.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.617+0000 7f3fca82d640 1 Processor -- start 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.617+0000 7f3fca82d640 1 -- start start 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fca82d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fca82d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4132280 con 0x7f3fc4134f00 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47772/0 (socket says 172.21.15.133:47772) 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fc3fff640 1 -- 172.21.15.133:0/203810848 learned_addr learned my addr 172.21.15.133:0/203810848 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fc3fff640 1 -- 172.21.15.133:0/203810848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc4121eb0 con 0x7f3fc4134f00 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.618+0000 7f3fc3fff640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 crc :-1 s=READY pgs=12321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3fb8014540 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3fb80146a0 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc4124e00 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3fb8014940 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc4124390 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.619+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f3fb8014af0 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fc4135310 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc0ff9640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 0x7f3fb003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7f3fb8019040 con 0x7f3fc4134f00 2024-04-02T14:28:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc0ff9640 1 --2- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 0x7f3fb0041890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 --> v2:172.21.15.133:6801/1793372331 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fb0041f20 con 0x7f3fb003f400 2024-04-02T14:28:02.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (crc 0 0 0) 0x7f3fb8014df0 con 0x7f3fc4134f00 2024-04-02T14:28:02.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.620+0000 7f3fc37fe640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 0x7f3fb003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.621+0000 7f3fc8dca640 1 --2- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 0x7f3fb0041890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T14:28:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.621+0000 7f3fc37fe640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 0x7f3fb003ae90 crc :-1 s=READY pgs=3802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.623+0000 7f3fc8dca640 1 --2- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 0x7f3fb0041890 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.624+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== osd.0 v2:172.21.15.133:6801/1793372331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f3fb0041f20 con 0x7f3fb003f400 2024-04-02T14:28:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.627+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 --> v2:172.21.15.133:6801/1793372331 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3fc41246e0 con 0x7f3fb003f400 2024-04-02T14:28:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fc0ff9640 1 -- 172.21.15.133:0/203810848 <== osd.0 v2:172.21.15.133:6801/1793372331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f3fc41246e0 con 0x7f3fb003f400 2024-04-02T14:28:02.636 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-02T14:28:02.636 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "" 2024-04-02T14:28:02.636 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-02T14:28:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 msgr2=0x7f3fb0041890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 0x7f3fb0041890 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 msgr2=0x7f3fb003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 0x7f3fb003ae90 crc :-1 s=READY pgs=3802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 msgr2=0x7f3fc4120d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 crc :-1 s=READY pgs=12321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 shutdown_connections 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> v2:172.21.15.133:6801/1793372331 conn(0x7f3fb003f400 0x7f3fb0041890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f3fb0038980 0x7f3fb003ae90 unknown :-1 s=CLOSED pgs=3802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 --2- 172.21.15.133:0/203810848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc4134f00 0x7f3fc4120d20 unknown :-1 s=CLOSED pgs=12321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 >> 172.21.15.133:0/203810848 conn(0x7f3fc40a4a90 msgr2=0x7f3fc40a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 shutdown_connections 2024-04-02T14:28:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.629+0000 7f3fca82d640 1 -- 172.21.15.133:0/203810848 wait complete. 2024-04-02T14:28:02.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-02T14:28:02.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.797+0000 7fc68a0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:02.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.803+0000 7fc68a0ad640 1 Processor -- start 2024-04-02T14:28:02.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.803+0000 7fc68a0ad640 1 -- start start 2024-04-02T14:28:02.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc68a0ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc6841355f0 0x7fc684137a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc68a0ad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc68409cc60 con 0x7fc68412f500 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc68a0ad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fc68409d8d0 con 0x7fc6841355f0 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc682ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc6841355f0 0x7fc684137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc6837fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc68412f500 0x7fc68412f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37022/0 (socket says 172.21.15.133:37022) 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.804+0000 7fc6837fe640 1 -- 172.21.15.133:0/277938895 learned_addr learned my addr 172.21.15.133:0/277938895 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc682ffd640 1 -- 172.21.15.133:0/277938895 >> v1:172.21.15.133:6789/0 conn(0x7fc68412f500 legacy=0x7fc68412f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc682ffd640 1 -- 172.21.15.133:0/277938895 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68409dac0 con 0x7fc6841355f0 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc6827fc640 1 -- 172.21.15.133:0/277938895 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314232238 0 0) 0x7fc68409cc60 con 0x7fc68412f500 2024-04-02T14:28:02.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc682ffd640 1 --2- 172.21.15.133:0/277938895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6841355f0 0x7fc684137a10 crc :-1 s=READY pgs=12323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de462e912a215f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc6827fc640 1 -- 172.21.15.133:0/277938895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc678009d90 con 0x7fc6841355f0 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc6827fc640 1 -- 172.21.15.133:0/277938895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc67800e230 con 0x7fc6841355f0 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.805+0000 7fc6827fc640 1 -- 172.21.15.133:0/277938895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc67800e4d0 con 0x7fc6841355f0 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 -- 172.21.15.133:0/277938895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6841355f0 msgr2=0x7fc684137a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/277938895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6841355f0 0x7fc684137a10 crc :-1 s=READY pgs=12323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 -- 172.21.15.133:0/277938895 shutdown_connections 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/277938895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6841355f0 0x7fc684137a10 unknown :-1 s=CLOSED pgs=12323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 -- 172.21.15.133:0/277938895 >> 172.21.15.133:0/277938895 conn(0x7fc6840a18a0 msgr2=0x7fc6840a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:02.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 -- 172.21.15.133:0/277938895 shutdown_connections 2024-04-02T14:28:02.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 1 -- 172.21.15.133:0/277938895 wait complete. 2024-04-02T14:28:02.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.806+0000 7fc68a0ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.807+0000 7fc68a0ad640 1 Processor -- start 2024-04-02T14:28:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.807+0000 7fc68a0ad640 1 -- start start 2024-04-02T14:28:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.808+0000 7fc68a0ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.808+0000 7fc68a0ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc68411e440 con 0x7fc68412f500 2024-04-02T14:28:02.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc6837fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc6837fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47786/0 (socket says 172.21.15.133:47786) 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc6837fe640 1 -- 172.21.15.133:0/1135975683 learned_addr learned my addr 172.21.15.133:0/1135975683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc6837fe640 1 -- 172.21.15.133:0/1135975683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68411ce30 con 0x7fc68412f500 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc6837fe640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 crc :-1 s=READY pgs=12324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc674004b80 con 0x7fc68412f500 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc674004ce0 con 0x7fc68412f500 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68411d590 con 0x7fc68412f500 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc674007120 con 0x7fc68412f500 2024-04-02T14:28:02.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.809+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68411ec50 con 0x7fc68412f500 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.810+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fc6740072d0 con 0x7fc68412f500 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.810+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6841c3cd0 con 0x7fc68412f500 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.810+0000 7fc66bfff640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 0x7fc67003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.810+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fc674048600 con 0x7fc68412f500 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.811+0000 7fc682ffd640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 0x7fc67003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:02.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.811+0000 7fc682ffd640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 0x7fc67003ae90 crc :-1 s=READY pgs=3803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:02.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.815+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fc67400a9e0 con 0x7fc68412f500 2024-04-02T14:28:02.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.994+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc684073690 con 0x7fc670038980 2024-04-02T14:28:02.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.995+0000 7fc66bfff640 1 -- 172.21.15.133:0/1135975683 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fc67800e3a0 con 0x7fc670038980 2024-04-02T14:28:02.997 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 0 to deep-scrub 2024-04-02T14:28:02.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 msgr2=0x7fc67003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:02.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 0x7fc67003ae90 crc :-1 s=READY pgs=3803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:02.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 msgr2=0x7fc6841c3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 crc :-1 s=READY pgs=12324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 shutdown_connections 2024-04-02T14:28:03.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc670038980 0x7fc67003ae90 unknown :-1 s=CLOSED pgs=3803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 --2- 172.21.15.133:0/1135975683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68412f500 0x7fc6841c3ec0 unknown :-1 s=CLOSED pgs=12324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 >> 172.21.15.133:0/1135975683 conn(0x7fc6840a18a0 msgr2=0x7fc68412fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 shutdown_connections 2024-04-02T14:28:03.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:02.997+0000 7fc68a0ad640 1 -- 172.21.15.133:0/1135975683 wait complete. 2024-04-02T14:28:03.012 INFO:tasks.ceph:Scrubbing osd.1 2024-04-02T14:28:03.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-02T14:28:03.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.125+0000 7fbbdc0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.134+0000 7fbbdc0ed640 1 Processor -- start 2024-04-02T14:28:03.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.134+0000 7fbbdc0ed640 1 -- start start 2024-04-02T14:28:03.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.135+0000 7fbbdc0ed640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbbd409ca10 0x7fbbd4130aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.135+0000 7fbbdc0ed640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd4130870 con 0x7fbbd4130fe0 2024-04-02T14:28:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.135+0000 7fbbdc0ed640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd406e490 con 0x7fbbd409ca10 2024-04-02T14:28:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9e89640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbbd409ca10 0x7fbbd4130aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9688640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbbd4130fe0 0x7fbbd412d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37036/0 (socket says 172.21.15.133:37036) 2024-04-02T14:28:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9688640 1 -- 172.21.15.133:0/3989105397 learned_addr learned my addr 172.21.15.133:0/3989105397 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9e89640 1 -- 172.21.15.133:0/3989105397 >> v1:172.21.15.133:6789/0 conn(0x7fbbd4130fe0 legacy=0x7fbbd412d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9e89640 1 -- 172.21.15.133:0/3989105397 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd409c510 con 0x7fbbd409ca10 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd9e89640 1 --2- 172.21.15.133:0/3989105397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd4130aa0 crc :-1 s=READY pgs=12326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff721a17ba59ab19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd8e87640 1 -- 172.21.15.133:0/3989105397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbbc4009d90 con 0x7fbbd409ca10 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd8e87640 1 -- 172.21.15.133:0/3989105397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbbc400e230 con 0x7fbbd409ca10 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.136+0000 7fbbd8e87640 1 -- 172.21.15.133:0/3989105397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbbc400e4d0 con 0x7fbbd409ca10 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/3989105397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 msgr2=0x7fbbd4130aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/3989105397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd4130aa0 crc :-1 s=READY pgs=12326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/3989105397 shutdown_connections 2024-04-02T14:28:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/3989105397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd4130aa0 unknown :-1 s=CLOSED pgs=12326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/3989105397 >> 172.21.15.133:0/3989105397 conn(0x7fbbd40a0680 msgr2=0x7fbbd40a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/3989105397 shutdown_connections 2024-04-02T14:28:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/3989105397 wait complete. 2024-04-02T14:28:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.137+0000 7fbbdc0ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.138+0000 7fbbdc0ed640 1 Processor -- start 2024-04-02T14:28:03.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.139+0000 7fbbdc0ed640 1 -- start start 2024-04-02T14:28:03.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.139+0000 7fbbdc0ed640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.139+0000 7fbbdc0ed640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41363b0 con 0x7fbbd409ca10 2024-04-02T14:28:03.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbd9e89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbd9e89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47810/0 (socket says 172.21.15.133:47810) 2024-04-02T14:28:03.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbd9e89640 1 -- 172.21.15.133:0/2703510736 learned_addr learned my addr 172.21.15.133:0/2703510736 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:03.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbd9e89640 1 -- 172.21.15.133:0/2703510736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4138b70 con 0x7fbbd409ca10 2024-04-02T14:28:03.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbd9e89640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 crc :-1 s=READY pgs=12327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbbc4014620 con 0x7fbbd409ca10 2024-04-02T14:28:03.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbbc4014780 con 0x7fbbd409ca10 2024-04-02T14:28:03.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd4138780 con 0x7fbbd409ca10 2024-04-02T14:28:03.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbbc4014a20 con 0x7fbbd409ca10 2024-04-02T14:28:03.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.140+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd4136be0 con 0x7fbbd409ca10 2024-04-02T14:28:03.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.141+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbd409d080 con 0x7fbbd409ca10 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.141+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fbbc4014c20 con 0x7fbbd409ca10 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.141+0000 7fbbca7fc640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 0x7fbbb803ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.141+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fbbc4019040 con 0x7fbbd409ca10 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbca7fc640 1 --2- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 0x7fbbb8041840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 --> v2:172.21.15.133:6803/4121831248 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbb8041ed0 con 0x7fbbb803f3b0 2024-04-02T14:28:03.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (crc 0 0 0) 0x7fbbc404c230 con 0x7fbbd409ca10 2024-04-02T14:28:03.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbd9688640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 0x7fbbb803ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbda68a640 1 --2- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 0x7fbbb8041840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T14:28:03.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.142+0000 7fbbd9688640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 0x7fbbb803ae40 crc :-1 s=READY pgs=3804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.145+0000 7fbbda68a640 1 --2- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 0x7fbbb8041840 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.146+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== osd.1 v2:172.21.15.133:6803/4121831248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7fbbb8041ed0 con 0x7fbbb803f3b0 2024-04-02T14:28:03.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.149+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 --> v2:172.21.15.133:6803/4121831248 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbbd4138910 con 0x7fbbb803f3b0 2024-04-02T14:28:03.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbca7fc640 1 -- 172.21.15.133:0/2703510736 <== osd.1 v2:172.21.15.133:6803/4121831248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7fbbd4138910 con 0x7fbbb803f3b0 2024-04-02T14:28:03.158 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-02T14:28:03.158 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "" 2024-04-02T14:28:03.159 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-02T14:28:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 msgr2=0x7fbbb8041840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 0x7fbbb8041840 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 msgr2=0x7fbbb803ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 0x7fbbb803ae40 crc :-1 s=READY pgs=3804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 msgr2=0x7fbbd41c3db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 crc :-1 s=READY pgs=12327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 shutdown_connections 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fbbb8038930 0x7fbbb803ae40 unknown :-1 s=CLOSED pgs=3804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> v2:172.21.15.133:6803/4121831248 conn(0x7fbbb803f3b0 0x7fbbb8041840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 --2- 172.21.15.133:0/2703510736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbd409ca10 0x7fbbd41c3db0 unknown :-1 s=CLOSED pgs=12327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 >> 172.21.15.133:0/2703510736 conn(0x7fbbd40a0680 msgr2=0x7fbbd409d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 shutdown_connections 2024-04-02T14:28:03.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.151+0000 7fbbdc0ed640 1 -- 172.21.15.133:0/2703510736 wait complete. 2024-04-02T14:28:03.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-02T14:28:03.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.270+0000 7fe22e39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.275+0000 7fe22e39d640 1 Processor -- start 2024-04-02T14:28:03.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.275+0000 7fe22e39d640 1 -- start start 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe22e39d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2281313d0 0x7fe228135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe22e39d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe228130a60 con 0x7fe22806e260 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe22e39d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fe22809c8b0 con 0x7fe2281313d0 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe2277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2281313d0 0x7fe228135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe227fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe22806e260 0x7fe228130cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37046/0 (socket says 172.21.15.133:37046) 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe227fff640 1 -- 172.21.15.133:0/642765224 learned_addr learned my addr 172.21.15.133:0/642765224 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:03.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe2277fe640 1 -- 172.21.15.133:0/642765224 >> v1:172.21.15.133:6789/0 conn(0x7fe22806e260 legacy=0x7fe228130cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe2277fe640 1 -- 172.21.15.133:0/642765224 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22809c2f0 con 0x7fe2281313d0 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe226ffd640 1 -- 172.21.15.133:0/642765224 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262053169 0 0) 0x7fe228130a60 con 0x7fe22806e260 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe2277fe640 1 --2- 172.21.15.133:0/642765224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2281313d0 0x7fe228135dc0 crc :-1 s=READY pgs=12329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c661172d0703935 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe226ffd640 1 -- 172.21.15.133:0/642765224 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe21c00ad20 con 0x7fe2281313d0 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe226ffd640 1 -- 172.21.15.133:0/642765224 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe21c00ae80 con 0x7fe2281313d0 2024-04-02T14:28:03.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.277+0000 7fe226ffd640 1 -- 172.21.15.133:0/642765224 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe21c00f470 con 0x7fe2281313d0 2024-04-02T14:28:03.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 -- 172.21.15.133:0/642765224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2281313d0 msgr2=0x7fe228135dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 --2- 172.21.15.133:0/642765224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2281313d0 0x7fe228135dc0 crc :-1 s=READY pgs=12329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 -- 172.21.15.133:0/642765224 shutdown_connections 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 --2- 172.21.15.133:0/642765224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2281313d0 0x7fe228135dc0 unknown :-1 s=CLOSED pgs=12329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 -- 172.21.15.133:0/642765224 >> 172.21.15.133:0/642765224 conn(0x7fe2280a0790 msgr2=0x7fe2280a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 -- 172.21.15.133:0/642765224 shutdown_connections 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 1 -- 172.21.15.133:0/642765224 wait complete. 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.278+0000 7fe22e39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.280+0000 7fe22e39d640 1 Processor -- start 2024-04-02T14:28:03.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.280+0000 7fe22e39d640 1 -- start start 2024-04-02T14:28:03.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe22e39d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe22e39d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe22806aad0 con 0x7fe22806e260 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe227fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe227fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47822/0 (socket says 172.21.15.133:47822) 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe227fff640 1 -- 172.21.15.133:0/3479416321 learned_addr learned my addr 172.21.15.133:0/3479416321 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe227fff640 1 -- 172.21.15.133:0/3479416321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22811b0a0 con 0x7fe22806e260 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe227fff640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 crc :-1 s=READY pgs=12330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe218004b80 con 0x7fe22806e260 2024-04-02T14:28:03.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe218004ce0 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.281+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22811b930 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.282+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe218007120 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.282+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22811c9a0 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.282+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22811d480 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.283+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fe218007320 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.283+0000 7fe224ff9640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 0x7fe21403ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.283+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fe218048720 con 0x7fe22806e260 2024-04-02T14:28:03.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.283+0000 7fe2277fe640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 0x7fe21403ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.284+0000 7fe2277fe640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 0x7fe21403ae40 crc :-1 s=READY pgs=3805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.288+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fe21800f230 con 0x7fe22806e260 2024-04-02T14:28:03.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.465+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe228119ad0 con 0x7fe214038930 2024-04-02T14:28:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.465+0000 7fe224ff9640 1 -- 172.21.15.133:0/3479416321 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fe228119ad0 con 0x7fe214038930 2024-04-02T14:28:03.467 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 1 to deep-scrub 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 msgr2=0x7fe21403ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 0x7fe21403ae40 crc :-1 s=READY pgs=3805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 msgr2=0x7fe228119cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 crc :-1 s=READY pgs=12330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 shutdown_connections 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe214038930 0x7fe21403ae40 unknown :-1 s=CLOSED pgs=3805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 --2- 172.21.15.133:0/3479416321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe22806e260 0x7fe228119cd0 unknown :-1 s=CLOSED pgs=12330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 >> 172.21.15.133:0/3479416321 conn(0x7fe2280a0790 msgr2=0x7fe22812f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 shutdown_connections 2024-04-02T14:28:03.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.468+0000 7fe22e39d640 1 -- 172.21.15.133:0/3479416321 wait complete. 2024-04-02T14:28:03.485 INFO:tasks.ceph:Scrubbing osd.2 2024-04-02T14:28:03.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-02T14:28:03.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.596+0000 7fadac86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.605+0000 7fadac86d640 1 Processor -- start 2024-04-02T14:28:03.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.605+0000 7fadac86d640 1 -- start start 2024-04-02T14:28:03.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fadac86d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fada812fcc0 0x7fada8137820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fadac86d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada8074ed0 con 0x7fada809c690 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fadac86d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fada809cdb0 con 0x7fada812fcc0 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fada5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fada812fcc0 0x7fada8137820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fada659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fada809c690 0x7fada812f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37048/0 (socket says 172.21.15.133:37048) 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.606+0000 7fada659c640 1 -- 172.21.15.133:0/1503743580 learned_addr learned my addr 172.21.15.133:0/1503743580 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada559a640 1 -- 172.21.15.133:0/1503743580 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721506785 0 0) 0x7fada8074ed0 con 0x7fada809c690 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada559a640 1 -- 172.21.15.133:0/1503743580 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad90003580 con 0x7fada809c690 2024-04-02T14:28:03.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada5d9b640 1 -- 172.21.15.133:0/1503743580 >> v1:172.21.15.133:6789/0 conn(0x7fada809c690 legacy=0x7fada812f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada5d9b640 1 -- 172.21.15.133:0/1503743580 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada809caa0 con 0x7fada812fcc0 2024-04-02T14:28:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada5d9b640 1 --2- 172.21.15.133:0/1503743580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada812fcc0 0x7fada8137820 crc :-1 s=READY pgs=12332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973d0469f0febe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada559a640 1 -- 172.21.15.133:0/1503743580 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fad9c009d90 con 0x7fada812fcc0 2024-04-02T14:28:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada559a640 1 -- 172.21.15.133:0/1503743580 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fad9c00e230 con 0x7fada812fcc0 2024-04-02T14:28:03.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.607+0000 7fada559a640 1 -- 172.21.15.133:0/1503743580 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fad9c00e4d0 con 0x7fada812fcc0 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 -- 172.21.15.133:0/1503743580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada812fcc0 msgr2=0x7fada8137820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 --2- 172.21.15.133:0/1503743580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada812fcc0 0x7fada8137820 crc :-1 s=READY pgs=12332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 -- 172.21.15.133:0/1503743580 shutdown_connections 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 --2- 172.21.15.133:0/1503743580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada812fcc0 0x7fada8137820 unknown :-1 s=CLOSED pgs=12332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 -- 172.21.15.133:0/1503743580 >> 172.21.15.133:0/1503743580 conn(0x7fada80a0b50 msgr2=0x7fada80a0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 -- 172.21.15.133:0/1503743580 shutdown_connections 2024-04-02T14:28:03.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 1 -- 172.21.15.133:0/1503743580 wait complete. 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.608+0000 7fadac86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.609+0000 7fadac86d640 1 Processor -- start 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.609+0000 7fadac86d640 1 -- start start 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.610+0000 7fadac86d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.610+0000 7fadac86d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fada812a8d0 con 0x7fada809c690 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fada659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fada659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47840/0 (socket says 172.21.15.133:47840) 2024-04-02T14:28:03.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fada659c640 1 -- 172.21.15.133:0/3669103592 learned_addr learned my addr 172.21.15.133:0/3669103592 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fada659c640 1 -- 172.21.15.133:0/3669103592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada81283c0 con 0x7fada809c690 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fada659c640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 crc :-1 s=READY pgs=12333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fad98005a60 con 0x7fada809c690 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fad98005bc0 con 0x7fada809c690 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada81280f0 con 0x7fada809c690 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fad98005e60 con 0x7fada809c690 2024-04-02T14:28:03.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.611+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada812a1e0 con 0x7fada809c690 2024-04-02T14:28:03.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.612+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fada809e420 con 0x7fada809c690 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.612+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fad980070b0 con 0x7fada809c690 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.612+0000 7fad96ffd640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 0x7fad8803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.612+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fad980482e0 con 0x7fada809c690 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fad96ffd640 1 --2- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 0x7fad88041890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 --> v2:172.21.15.138:6805/1859851880 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad88041f20 con 0x7fad8803f400 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (crc 0 0 0) 0x7fad9804a7d0 con 0x7fada809c690 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fada5d9b640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 0x7fad8803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fada5d9b640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 0x7fad8803ae90 crc :-1 s=READY pgs=3806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.613+0000 7fada6d9d640 1 --2- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 0x7fad88041890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T14:28:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.616+0000 7fada6d9d640 1 --2- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 0x7fad88041890 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.618+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== osd.2 v2:172.21.15.138:6805/1859851880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7fad88041f20 con 0x7fad8803f400 2024-04-02T14:28:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.621+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 --> v2:172.21.15.138:6805/1859851880 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fada8137d60 con 0x7fad8803f400 2024-04-02T14:28:03.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.623+0000 7fad96ffd640 1 -- 172.21.15.133:0/3669103592 <== osd.2 v2:172.21.15.138:6805/1859851880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7fada000a820 con 0x7fad8803f400 2024-04-02T14:28:03.626 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-02T14:28:03.626 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "" 2024-04-02T14:28:03.627 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-02T14:28:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 msgr2=0x7fad88041890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 0x7fad88041890 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 msgr2=0x7fad8803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 0x7fad8803ae90 crc :-1 s=READY pgs=3806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 msgr2=0x7fada8128780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 crc :-1 s=READY pgs=12333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 shutdown_connections 2024-04-02T14:28:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> v2:172.21.15.138:6805/1859851880 conn(0x7fad8803f400 0x7fad88041890 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fad88038980 0x7fad8803ae90 unknown :-1 s=CLOSED pgs=3806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 --2- 172.21.15.133:0/3669103592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fada809c690 0x7fada8128780 unknown :-1 s=CLOSED pgs=12333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 >> 172.21.15.133:0/3669103592 conn(0x7fada80a0b50 msgr2=0x7fada80708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 shutdown_connections 2024-04-02T14:28:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.624+0000 7fadac86d640 1 -- 172.21.15.133:0/3669103592 wait complete. 2024-04-02T14:28:03.636 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-02T14:28:03.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.741+0000 7fe73bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.747+0000 7fe73bd9d640 1 Processor -- start 2024-04-02T14:28:03.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.747+0000 7fe73bd9d640 1 -- start start 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe73bd9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe73406e260 0x7fe734130cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe73bd9d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe734130a60 con 0x7fe734131200 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe73bd9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fe73409c8b0 con 0x7fe73406e260 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe739b39640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe73406e260 0x7fe734130cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe739338640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe734131200 0x7fe73409b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37064/0 (socket says 172.21.15.133:37064) 2024-04-02T14:28:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.748+0000 7fe739338640 1 -- 172.21.15.133:0/2068979793 learned_addr learned my addr 172.21.15.133:0/2068979793 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe739b39640 1 -- 172.21.15.133:0/2068979793 >> v1:172.21.15.133:6789/0 conn(0x7fe734131200 legacy=0x7fe73409b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe738b37640 1 -- 172.21.15.133:0/2068979793 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217105922 0 0) 0x7fe734130a60 con 0x7fe734131200 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe739b39640 1 -- 172.21.15.133:0/2068979793 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73409c2f0 con 0x7fe73406e260 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe739b39640 1 --2- 172.21.15.133:0/2068979793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe734130cc0 crc :-1 s=READY pgs=12335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efe11aa787dc96a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe738b37640 1 -- 172.21.15.133:0/2068979793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe72400e040 con 0x7fe73406e260 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe738b37640 1 -- 172.21.15.133:0/2068979793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe724007670 con 0x7fe73406e260 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.749+0000 7fe738b37640 1 -- 172.21.15.133:0/2068979793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe724007910 con 0x7fe73406e260 2024-04-02T14:28:03.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 -- 172.21.15.133:0/2068979793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 msgr2=0x7fe734130cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/2068979793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe734130cc0 crc :-1 s=READY pgs=12335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 -- 172.21.15.133:0/2068979793 shutdown_connections 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/2068979793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe734130cc0 unknown :-1 s=CLOSED pgs=12335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 -- 172.21.15.133:0/2068979793 >> 172.21.15.133:0/2068979793 conn(0x7fe7340a0790 msgr2=0x7fe7340a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 -- 172.21.15.133:0/2068979793 shutdown_connections 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 1 -- 172.21.15.133:0/2068979793 wait complete. 2024-04-02T14:28:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.750+0000 7fe73bd9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:03.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.751+0000 7fe73bd9d640 1 Processor -- start 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.751+0000 7fe73bd9d640 1 -- start start 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.752+0000 7fe73bd9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.752+0000 7fe73bd9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe73411c910 con 0x7fe73406e260 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe739b39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe739b39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47848/0 (socket says 172.21.15.133:47848) 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe739b39640 1 -- 172.21.15.133:0/3277712124 learned_addr learned my addr 172.21.15.133:0/3277712124 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe739b39640 1 -- 172.21.15.133:0/3277712124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73412ae40 con 0x7fe73406e260 2024-04-02T14:28:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe739b39640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 crc :-1 s=READY pgs=12336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe72400e040 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe724012b00 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73411f720 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe724012da0 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.753+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73411e020 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.754+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fe7240153f0 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.754+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73411db80 con 0x7fe73406e260 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.754+0000 7fe7327fc640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 0x7fe72003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:03.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.755+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fe724049dc0 con 0x7fe73406e260 2024-04-02T14:28:03.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.755+0000 7fe739338640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 0x7fe72003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:03.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.755+0000 7fe739338640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 0x7fe72003ae90 crc :-1 s=READY pgs=3807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:03.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.760+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fe72400fc80 con 0x7fe73406e260 2024-04-02T14:28:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.936+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fe734063400 con 0x7fe720038980 2024-04-02T14:28:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.936+0000 7fe7327fc640 1 -- 172.21.15.133:0/3277712124 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fe734063400 con 0x7fe720038980 2024-04-02T14:28:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 2 to deep-scrub 2024-04-02T14:28:03.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 msgr2=0x7fe72003ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 0x7fe72003ae90 crc :-1 s=READY pgs=3807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 msgr2=0x7fe73411ccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 crc :-1 s=READY pgs=12336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 shutdown_connections 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fe720038980 0x7fe72003ae90 unknown :-1 s=CLOSED pgs=3807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 --2- 172.21.15.133:0/3277712124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe73406e260 0x7fe73411ccd0 unknown :-1 s=CLOSED pgs=12336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 >> 172.21.15.133:0/3277712124 conn(0x7fe7340a0790 msgr2=0x7fe73412d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 shutdown_connections 2024-04-02T14:28:03.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:03.939+0000 7fe73bd9d640 1 -- 172.21.15.133:0/3277712124 wait complete. 2024-04-02T14:28:03.951 INFO:tasks.ceph:Scrubbing osd.3 2024-04-02T14:28:03.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-02T14:28:04.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.059+0000 7f7aaf424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.068+0000 7f7aaf424640 1 Processor -- start 2024-04-02T14:28:04.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.068+0000 7f7aaf424640 1 -- start start 2024-04-02T14:28:04.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aaf424640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7aa81312e0 0x7f7aa81316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aaf424640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa809dca0 con 0x7f7aa8131c30 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aaf424640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa809cef0 con 0x7f7aa81312e0 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aad1c0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7aa81312e0 0x7f7aa81316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aac9bf640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7aa8131c30 0x7f7aa812e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37068/0 (socket says 172.21.15.133:37068) 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.069+0000 7f7aac9bf640 1 -- 172.21.15.133:0/775458388 learned_addr learned my addr 172.21.15.133:0/775458388 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7aad1c0640 1 -- 172.21.15.133:0/775458388 >> v1:172.21.15.133:6789/0 conn(0x7f7aa8131c30 legacy=0x7f7aa812e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7aad1c0640 1 -- 172.21.15.133:0/775458388 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa809cbe0 con 0x7f7aa81312e0 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7a9ffff640 1 -- 172.21.15.133:0/775458388 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374859469 0 0) 0x7f7aa809dca0 con 0x7f7aa8131c30 2024-04-02T14:28:04.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7aad1c0640 1 --2- 172.21.15.133:0/775458388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa81316f0 crc :-1 s=READY pgs=12338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a770232ecbb16d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7a9ffff640 1 -- 172.21.15.133:0/775458388 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7aa4009d90 con 0x7f7aa81312e0 2024-04-02T14:28:04.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7a9ffff640 1 -- 172.21.15.133:0/775458388 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7aa400e230 con 0x7f7aa81312e0 2024-04-02T14:28:04.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.070+0000 7f7a9ffff640 1 -- 172.21.15.133:0/775458388 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7aa400e4d0 con 0x7f7aa81312e0 2024-04-02T14:28:04.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 -- 172.21.15.133:0/775458388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 msgr2=0x7f7aa81316f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 --2- 172.21.15.133:0/775458388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa81316f0 crc :-1 s=READY pgs=12338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 -- 172.21.15.133:0/775458388 shutdown_connections 2024-04-02T14:28:04.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 --2- 172.21.15.133:0/775458388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa81316f0 unknown :-1 s=CLOSED pgs=12338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 -- 172.21.15.133:0/775458388 >> 172.21.15.133:0/775458388 conn(0x7f7aa80a17a0 msgr2=0x7f7aa80a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 -- 172.21.15.133:0/775458388 shutdown_connections 2024-04-02T14:28:04.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 1 -- 172.21.15.133:0/775458388 wait complete. 2024-04-02T14:28:04.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.071+0000 7f7aaf424640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.072+0000 7f7aaf424640 1 Processor -- start 2024-04-02T14:28:04.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.072+0000 7f7aaf424640 1 -- start start 2024-04-02T14:28:04.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.073+0000 7f7aaf424640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.073+0000 7f7aaf424640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa811f0f0 con 0x7f7aa81312e0 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aad1c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aad1c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47862/0 (socket says 172.21.15.133:47862) 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aad1c0640 1 -- 172.21.15.133:0/2819932058 learned_addr learned my addr 172.21.15.133:0/2819932058 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aad1c0640 1 -- 172.21.15.133:0/2819932058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa806dfc0 con 0x7f7aa81312e0 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aad1c0640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 crc :-1 s=READY pgs=12339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7aa4013610 con 0x7f7aa81312e0 2024-04-02T14:28:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7aa4013770 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa811fb70 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7aa4013a10 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.074+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa811e5c0 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.075+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f7aa4013bc0 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.075+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7aa811ef80 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.075+0000 7f7a9dffb640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 0x7f7a8c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.075+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7f7aa4018040 con 0x7f7aa81312e0 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.075+0000 7f7a9dffb640 1 --2- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 0x7f7a8c041890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.076+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 --> v2:172.21.15.138:6806/3056026735 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a8c041f20 con 0x7f7a8c03f400 2024-04-02T14:28:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.076+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=287) ==== 24+0+0 (crc 0 0 0) 0x7f7aa4049f60 con 0x7f7aa81312e0 2024-04-02T14:28:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.076+0000 7f7aac9bf640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 0x7f7a8c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.076+0000 7f7aac9bf640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 0x7f7a8c03ae90 crc :-1 s=READY pgs=3808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.076+0000 7f7aad9c1640 1 --2- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 0x7f7a8c041890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-04-02T14:28:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.079+0000 7f7aad9c1640 1 --2- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 0x7f7a8c041890 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.080+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== osd.3 v2:172.21.15.138:6806/3056026735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5172 (crc 0 0 0) 0x7f7a8c041f20 con 0x7f7a8c03f400 2024-04-02T14:28:04.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.083+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 --> v2:172.21.15.138:6806/3056026735 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7aa8131e80 con 0x7f7a8c03f400 2024-04-02T14:28:04.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.085+0000 7f7a9dffb640 1 -- 172.21.15.133:0/2819932058 <== osd.3 v2:172.21.15.138:6806/3056026735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f7aa8131e80 con 0x7f7a8c03f400 2024-04-02T14:28:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-02T14:28:04.087 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "" 2024-04-02T14:28:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-02T14:28:04.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.085+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 msgr2=0x7f7a8c041890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.085+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 0x7f7a8c041890 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 msgr2=0x7f7a8c03ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 0x7f7a8c03ae90 crc :-1 s=READY pgs=3808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 msgr2=0x7f7aa811cbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 crc :-1 s=READY pgs=12339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 shutdown_connections 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> v2:172.21.15.138:6806/3056026735 conn(0x7f7a8c03f400 0x7f7a8c041890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f7a8c038980 0x7f7a8c03ae90 unknown :-1 s=CLOSED pgs=3808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 --2- 172.21.15.133:0/2819932058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7aa81312e0 0x7f7aa811cbc0 unknown :-1 s=CLOSED pgs=12339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 >> 172.21.15.133:0/2819932058 conn(0x7f7aa80a17a0 msgr2=0x7f7aa81b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 shutdown_connections 2024-04-02T14:28:04.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.086+0000 7f7aaf424640 1 -- 172.21.15.133:0/2819932058 wait complete. 2024-04-02T14:28:04.097 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-02T14:28:04.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.203+0000 7ff812d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.209+0000 7ff812d9d640 1 Processor -- start 2024-04-02T14:28:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.209+0000 7ff812d9d640 1 -- start start 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff812d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff80c131780 0x7ff80c131b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff812d9d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff80c0a6af0 con 0x7ff80c09edf0 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff812d9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7ff80c073660 con 0x7ff80c131780 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff810b39640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff80c131780 0x7ff80c131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff80c09edf0 0x7ff80c09f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37070/0 (socket says 172.21.15.133:37070) 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80bfff640 1 -- 172.21.15.133:0/2921887486 learned_addr learned my addr 172.21.15.133:0/2921887486 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff810b39640 1 -- 172.21.15.133:0/2921887486 >> v1:172.21.15.133:6789/0 conn(0x7ff80c09edf0 legacy=0x7ff80c09f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff810b39640 1 -- 172.21.15.133:0/2921887486 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c0a67e0 con 0x7ff80c131780 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80b7fe640 1 -- 172.21.15.133:0/2921887486 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221320083 0 0) 0x7ff80c0a6af0 con 0x7ff80c09edf0 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff810b39640 1 --2- 172.21.15.133:0/2921887486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c131780 0x7ff80c131b90 crc :-1 s=READY pgs=12341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf4a126e32e3aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80b7fe640 1 -- 172.21.15.133:0/2921887486 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7fc009d90 con 0x7ff80c131780 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80b7fe640 1 -- 172.21.15.133:0/2921887486 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff7fc00e230 con 0x7ff80c131780 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.210+0000 7ff80b7fe640 1 -- 172.21.15.133:0/2921887486 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7fc00e4d0 con 0x7ff80c131780 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 -- 172.21.15.133:0/2921887486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c131780 msgr2=0x7ff80c131b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2921887486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c131780 0x7ff80c131b90 crc :-1 s=READY pgs=12341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 -- 172.21.15.133:0/2921887486 shutdown_connections 2024-04-02T14:28:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2921887486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c131780 0x7ff80c131b90 unknown :-1 s=CLOSED pgs=12341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 -- 172.21.15.133:0/2921887486 >> 172.21.15.133:0/2921887486 conn(0x7ff80c0a4ba0 msgr2=0x7ff80c0a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 -- 172.21.15.133:0/2921887486 shutdown_connections 2024-04-02T14:28:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 1 -- 172.21.15.133:0/2921887486 wait complete. 2024-04-02T14:28:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.211+0000 7ff812d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.213+0000 7ff812d9d640 1 Processor -- start 2024-04-02T14:28:04.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.213+0000 7ff812d9d640 1 -- start start 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff812d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff812d9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1204d0 con 0x7ff80c09edf0 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff810b39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff810b39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47878/0 (socket says 172.21.15.133:47878) 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff810b39640 1 -- 172.21.15.133:0/2024187617 learned_addr learned my addr 172.21.15.133:0/2024187617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff810b39640 1 -- 172.21.15.133:0/2024187617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c120760 con 0x7ff80c09edf0 2024-04-02T14:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff810b39640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 crc :-1 s=READY pgs=12342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.214+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7fc013610 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.215+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff7fc013770 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.215+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c121460 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.215+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff7fc013a10 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.215+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c121fc0 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.215+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c11f490 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.216+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7ff7fc013bc0 con 0x7ff80c09edf0 2024-04-02T14:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.216+0000 7ff8097fa640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 0x7ff7f803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.216+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7ff7fc018040 con 0x7ff80c09edf0 2024-04-02T14:28:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.216+0000 7ff80bfff640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 0x7ff7f803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.217+0000 7ff80bfff640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 0x7ff7f803ae90 crc :-1 s=READY pgs=3809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.221+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7ff7fc018c70 con 0x7ff80c09edf0 2024-04-02T14:28:04.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.399+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ff80c06e4c0 con 0x7ff7f8038980 2024-04-02T14:28:04.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.399+0000 7ff8097fa640 1 -- 172.21.15.133:0/2024187617 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7ff8000070b0 con 0x7ff7f8038980 2024-04-02T14:28:04.401 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 3 to deep-scrub 2024-04-02T14:28:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.401+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 msgr2=0x7ff7f803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.401+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 0x7ff7f803ae90 crc :-1 s=READY pgs=3809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.401+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 msgr2=0x7ff80c11df60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.401+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 crc :-1 s=READY pgs=12342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 shutdown_connections 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7ff7f8038980 0x7ff7f803ae90 unknown :-1 s=CLOSED pgs=3809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 --2- 172.21.15.133:0/2024187617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80c09edf0 0x7ff80c11df60 unknown :-1 s=CLOSED pgs=12342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 >> 172.21.15.133:0/2024187617 conn(0x7ff80c0a4ba0 msgr2=0x7ff80c133470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 shutdown_connections 2024-04-02T14:28:04.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.402+0000 7ff812d9d640 1 -- 172.21.15.133:0/2024187617 wait complete. 2024-04-02T14:28:04.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:28:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.524+0000 7fc68cd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.529+0000 7fc68cd5d640 1 Processor -- start 2024-04-02T14:28:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.529+0000 7fc68cd5d640 1 -- start start 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68cd5d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc68809bd80 0x7fc68809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68cd5d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc688073440 con 0x7fc68809af20 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68cd5d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fc688070820 con 0x7fc68809bd80 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc68809bd80 0x7fc68809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc67ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc68809af20 0x7fc68812f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56372/0 (socket says 172.21.15.133:56372) 2024-04-02T14:28:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc67ffff640 1 -- 172.21.15.133:0/1219767410 learned_addr learned my addr 172.21.15.133:0/1219767410 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68659c640 1 -- 172.21.15.133:0/1219767410 >> v1:172.21.15.133:6789/0 conn(0x7fc68809af20 legacy=0x7fc68812f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.530+0000 7fc68659c640 1 -- 172.21.15.133:0/1219767410 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc688070510 con 0x7fc68809bd80 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68659c640 1 --2- 172.21.15.133:0/1219767410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809bd80 0x7fc68809a9e0 crc :-1 s=READY pgs=12344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7736ca927f814860 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc685d9b640 1 -- 172.21.15.133:0/1219767410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc678009d90 con 0x7fc68809bd80 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc685d9b640 1 -- 172.21.15.133:0/1219767410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc67800e230 con 0x7fc68809bd80 2024-04-02T14:28:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc685d9b640 1 -- 172.21.15.133:0/1219767410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc67800e4d0 con 0x7fc68809bd80 2024-04-02T14:28:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68cd5d640 1 -- 172.21.15.133:0/1219767410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809bd80 msgr2=0x7fc68809a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/1219767410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809bd80 0x7fc68809a9e0 crc :-1 s=READY pgs=12344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68cd5d640 1 -- 172.21.15.133:0/1219767410 shutdown_connections 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/1219767410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809bd80 0x7fc68809a9e0 unknown :-1 s=CLOSED pgs=12344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.531+0000 7fc68cd5d640 1 -- 172.21.15.133:0/1219767410 >> 172.21.15.133:0/1219767410 conn(0x7fc6880a0790 msgr2=0x7fc6880a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.532+0000 7fc68cd5d640 1 -- 172.21.15.133:0/1219767410 shutdown_connections 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.532+0000 7fc68cd5d640 1 -- 172.21.15.133:0/1219767410 wait complete. 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.532+0000 7fc68cd5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.533+0000 7fc68cd5d640 1 Processor -- start 2024-04-02T14:28:04.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.533+0000 7fc68cd5d640 1 -- start start 2024-04-02T14:28:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.534+0000 7fc68cd5d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.534+0000 7fc68cd5d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc68811ea20 con 0x7fc68809af20 2024-04-02T14:28:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.534+0000 7fc68659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.534+0000 7fc68659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54674/0 (socket says 172.21.15.133:54674) 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.534+0000 7fc68659c640 1 -- 172.21.15.133:0/3537465130 learned_addr learned my addr 172.21.15.133:0/3537465130 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc68659c640 1 -- 172.21.15.133:0/3537465130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68811e780 con 0x7fc68809af20 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc68659c640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 crc :-1 s=READY pgs=12345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc678014540 con 0x7fc68809af20 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc6780146a0 con 0x7fc68809af20 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68811d820 con 0x7fc68809af20 2024-04-02T14:28:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc678014940 con 0x7fc68809af20 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.535+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68812fcb0 con 0x7fc68809af20 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.536+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc688063400 con 0x7fc68809af20 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.536+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fc678014af0 con 0x7fc68809af20 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.536+0000 7fc67effd640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 0x7fc66803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.536+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fc678019040 con 0x7fc68809af20 2024-04-02T14:28:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.537+0000 7fc67ffff640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 0x7fc66803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:04.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.537+0000 7fc67ffff640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 0x7fc66803ae90 crc :-1 s=READY pgs=3810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.541+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fc67807bac0 con 0x7fc68809af20 2024-04-02T14:28:04.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.713+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc68806c950 con 0x7fc668038980 2024-04-02T14:28:04.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.719+0000 7fc67effd640 1 -- 172.21.15.133:0/3537465130 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+187053 (crc 0 0 0) 0x7fc68806c950 con 0x7fc668038980 2024-04-02T14:28:04.722 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:04.723 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T14:28:04.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 msgr2=0x7fc66803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 0x7fc66803ae90 crc :-1 s=READY pgs=3810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 msgr2=0x7fc68811bc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 crc :-1 s=READY pgs=12345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 shutdown_connections 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fc668038980 0x7fc66803ae90 unknown :-1 s=CLOSED pgs=3810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 --2- 172.21.15.133:0/3537465130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc68809af20 0x7fc68811bc50 unknown :-1 s=CLOSED pgs=12345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 >> 172.21.15.133:0/3537465130 conn(0x7fc6880a0790 msgr2=0x7fc68809d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 shutdown_connections 2024-04-02T14:28:04.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:04.722+0000 7fc68cd5d640 1 -- 172.21.15.133:0/3537465130 wait complete. 2024-04-02T14:28:04.735 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5715,"stamp":"2024-04-02T14:28:03.856297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594469,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":145,"num_write_kb":1905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":170848,"kb_used_data":6144,"kb_used_omap":0,"kb_used_meta":164672,"kb_avail":419259552,"statfs":{"total":429496729600,"available":429321781248,"internally_reserved":0,"allocated":6291456,"data_stored":2983642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":168624128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003679"},"pg_stats":[{"pgid":"86.2","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.060011+0000","last_change":"2024-04-02T11:33:41.348309+0000","last_active":"2024-04-02T11:33:44.060011+0000","last_peered":"2024-04-02T11:33:44.060011+0000","last_clean":"2024-04-02T11:33:44.060011+0000","last_became_active":"2024-04-02T11:33:41.342906+0000","last_became_peered":"2024-04-02T11:33:41.342906+0000","last_unstale":"2024-04-02T11:33:44.060011+0000","last_undegraded":"2024-04-02T11:33:44.060011+0000","last_fullsized":"2024-04-02T11:33:44.060011+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1f","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.083353+0000","last_change":"2024-04-02T11:32:04.380690+0000","last_active":"2024-04-02T11:33:44.083353+0000","last_peered":"2024-04-02T11:33:44.083353+0000","last_clean":"2024-04-02T11:33:44.083353+0000","last_became_active":"2024-04-02T11:32:04.372115+0000","last_became_peered":"2024-04-02T11:32:04.372115+0000","last_unstale":"2024-04-02T11:33:44.083353+0000","last_undegraded":"2024-04-02T11:33:44.083353+0000","last_fullsized":"2024-04-02T11:33:44.083353+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.3","version":"285'1","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.111981+0000","last_change":"2024-04-02T11:33:41.302524+0000","last_active":"2024-04-02T11:33:44.111981+0000","last_peered":"2024-04-02T11:33:44.111981+0000","last_clean":"2024-04-02T11:33:44.111981+0000","last_became_active":"2024-04-02T11:33:41.286642+0000","last_became_peered":"2024-04-02T11:33:41.286642+0000","last_unstale":"2024-04-02T11:33:44.111981+0000","last_undegraded":"2024-04-02T11:33:44.111981+0000","last_fullsized":"2024-04-02T11:33:44.111981+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.479267+0000","last_change":"2024-04-02T11:32:04.439717+0000","last_active":"2024-04-02T11:33:44.479267+0000","last_peered":"2024-04-02T11:33:44.479267+0000","last_clean":"2024-04-02T11:33:44.479267+0000","last_became_active":"2024-04-02T11:32:04.424122+0000","last_became_peered":"2024-04-02T11:32:04.424122+0000","last_unstale":"2024-04-02T11:33:44.479267+0000","last_undegraded":"2024-04-02T11:33:44.479267+0000","last_fullsized":"2024-04-02T11:33:44.479267+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.0","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.109001+0000","last_change":"2024-04-02T11:33:41.305416+0000","last_active":"2024-04-02T11:33:44.109001+0000","last_peered":"2024-04-02T11:33:44.109001+0000","last_clean":"2024-04-02T11:33:44.109001+0000","last_became_active":"2024-04-02T11:33:41.268384+0000","last_became_peered":"2024-04-02T11:33:41.268384+0000","last_unstale":"2024-04-02T11:33:44.109001+0000","last_undegraded":"2024-04-02T11:33:44.109001+0000","last_fullsized":"2024-04-02T11:33:44.109001+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.608850+0000","last_change":"2024-04-02T11:32:04.410624+0000","last_active":"2024-04-02T11:33:44.608850+0000","last_peered":"2024-04-02T11:33:44.608850+0000","last_clean":"2024-04-02T11:33:44.608850+0000","last_became_active":"2024-04-02T11:32:04.394162+0000","last_became_peered":"2024-04-02T11:32:04.394162+0000","last_unstale":"2024-04-02T11:33:44.608850+0000","last_undegraded":"2024-04-02T11:33:44.608850+0000","last_fullsized":"2024-04-02T11:33:44.608850+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.1","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773986+0000","last_change":"2024-04-02T11:33:41.330641+0000","last_active":"2024-04-02T11:33:44.773986+0000","last_peered":"2024-04-02T11:33:44.773986+0000","last_clean":"2024-04-02T11:33:44.773986+0000","last_became_active":"2024-04-02T11:33:41.317736+0000","last_became_peered":"2024-04-02T11:33:41.317736+0000","last_unstale":"2024-04-02T11:33:44.773986+0000","last_undegraded":"2024-04-02T11:33:44.773986+0000","last_fullsized":"2024-04-02T11:33:44.773986+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.771453+0000","last_change":"2024-04-02T11:32:04.455109+0000","last_active":"2024-04-02T11:33:44.771453+0000","last_peered":"2024-04-02T11:33:44.771453+0000","last_clean":"2024-04-02T11:33:44.771453+0000","last_became_active":"2024-04-02T11:32:04.430454+0000","last_became_peered":"2024-04-02T11:32:04.430454+0000","last_unstale":"2024-04-02T11:33:44.771453+0000","last_undegraded":"2024-04-02T11:33:44.771453+0000","last_fullsized":"2024-04-02T11:33:44.771453+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.6","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.574697+0000","last_change":"2024-04-02T11:33:41.252370+0000","last_active":"2024-04-02T11:33:44.574697+0000","last_peered":"2024-04-02T11:33:44.574697+0000","last_clean":"2024-04-02T11:33:44.574697+0000","last_became_active":"2024-04-02T11:33:41.236833+0000","last_became_peered":"2024-04-02T11:33:41.236833+0000","last_unstale":"2024-04-02T11:33:44.574697+0000","last_undegraded":"2024-04-02T11:33:44.574697+0000","last_fullsized":"2024-04-02T11:33:44.574697+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.115811+0000","last_change":"2024-04-02T11:32:04.397555+0000","last_active":"2024-04-02T11:33:44.115811+0000","last_peered":"2024-04-02T11:33:44.115811+0000","last_clean":"2024-04-02T11:33:44.115811+0000","last_became_active":"2024-04-02T11:32:04.392132+0000","last_became_peered":"2024-04-02T11:32:04.392132+0000","last_unstale":"2024-04-02T11:33:44.115811+0000","last_undegraded":"2024-04-02T11:33:44.115811+0000","last_fullsized":"2024-04-02T11:33:44.115811+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.7","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.708501+0000","last_change":"2024-04-02T11:33:41.301764+0000","last_active":"2024-04-02T11:33:44.708501+0000","last_peered":"2024-04-02T11:33:44.708501+0000","last_clean":"2024-04-02T11:33:44.708501+0000","last_became_active":"2024-04-02T11:33:41.288766+0000","last_became_peered":"2024-04-02T11:33:41.288766+0000","last_unstale":"2024-04-02T11:33:44.708501+0000","last_undegraded":"2024-04-02T11:33:44.708501+0000","last_fullsized":"2024-04-02T11:33:44.708501+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.464372+0000","last_change":"2024-04-02T11:32:04.448308+0000","last_active":"2024-04-02T11:33:44.464372+0000","last_peered":"2024-04-02T11:33:44.464372+0000","last_clean":"2024-04-02T11:33:44.464372+0000","last_became_active":"2024-04-02T11:32:04.435642+0000","last_became_peered":"2024-04-02T11:32:04.435642+0000","last_unstale":"2024-04-02T11:33:44.464372+0000","last_undegraded":"2024-04-02T11:33:44.464372+0000","last_fullsized":"2024-04-02T11:33:44.464372+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.4","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.746493+0000","last_change":"2024-04-02T11:33:41.237088+0000","last_active":"2024-04-02T11:33:44.746493+0000","last_peered":"2024-04-02T11:33:44.746493+0000","last_clean":"2024-04-02T11:33:44.746493+0000","last_became_active":"2024-04-02T11:33:41.230755+0000","last_became_peered":"2024-04-02T11:33:41.230755+0000","last_unstale":"2024-04-02T11:33:44.746493+0000","last_undegraded":"2024-04-02T11:33:44.746493+0000","last_fullsized":"2024-04-02T11:33:44.746493+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.476796+0000","last_change":"2024-04-02T11:32:04.531627+0000","last_active":"2024-04-02T11:33:44.476796+0000","last_peered":"2024-04-02T11:33:44.476796+0000","last_clean":"2024-04-02T11:33:44.476796+0000","last_became_active":"2024-04-02T11:32:04.523227+0000","last_became_peered":"2024-04-02T11:32:04.523227+0000","last_unstale":"2024-04-02T11:33:44.476796+0000","last_undegraded":"2024-04-02T11:33:44.476796+0000","last_fullsized":"2024-04-02T11:33:44.476796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.5","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.613522+0000","last_change":"2024-04-02T11:33:41.243595+0000","last_active":"2024-04-02T11:33:44.613522+0000","last_peered":"2024-04-02T11:33:44.613522+0000","last_clean":"2024-04-02T11:33:44.613522+0000","last_became_active":"2024-04-02T11:33:41.229522+0000","last_became_peered":"2024-04-02T11:33:41.229522+0000","last_unstale":"2024-04-02T11:33:44.613522+0000","last_undegraded":"2024-04-02T11:33:44.613522+0000","last_fullsized":"2024-04-02T11:33:44.613522+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.508048+0000","last_change":"2024-04-02T11:32:04.499203+0000","last_active":"2024-04-02T11:33:44.508048+0000","last_peered":"2024-04-02T11:33:44.508048+0000","last_clean":"2024-04-02T11:33:44.508048+0000","last_became_active":"2024-04-02T11:32:04.490360+0000","last_became_peered":"2024-04-02T11:32:04.490360+0000","last_unstale":"2024-04-02T11:33:44.508048+0000","last_undegraded":"2024-04-02T11:33:44.508048+0000","last_fullsized":"2024-04-02T11:33:44.508048+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.619749+0000","last_change":"2024-04-02T11:33:41.308637+0000","last_active":"2024-04-02T11:33:44.619749+0000","last_peered":"2024-04-02T11:33:44.619749+0000","last_clean":"2024-04-02T11:33:44.619749+0000","last_became_active":"2024-04-02T11:33:41.301597+0000","last_became_peered":"2024-04-02T11:33:41.301597+0000","last_unstale":"2024-04-02T11:33:44.619749+0000","last_undegraded":"2024-04-02T11:33:44.619749+0000","last_fullsized":"2024-04-02T11:33:44.619749+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.751562+0000","last_change":"2024-04-02T11:32:04.460524+0000","last_active":"2024-04-02T11:33:44.751562+0000","last_peered":"2024-04-02T11:33:44.751562+0000","last_clean":"2024-04-02T11:33:44.751562+0000","last_became_active":"2024-04-02T11:32:04.427391+0000","last_became_peered":"2024-04-02T11:32:04.427391+0000","last_unstale":"2024-04-02T11:33:44.751562+0000","last_undegraded":"2024-04-02T11:33:44.751562+0000","last_fullsized":"2024-04-02T11:33:44.751562+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.087235+0000","last_change":"2024-04-02T11:33:41.282206+0000","last_active":"2024-04-02T11:33:44.087235+0000","last_peered":"2024-04-02T11:33:44.087235+0000","last_clean":"2024-04-02T11:33:44.087235+0000","last_became_active":"2024-04-02T11:33:41.264133+0000","last_became_peered":"2024-04-02T11:33:41.264133+0000","last_unstale":"2024-04-02T11:33:44.087235+0000","last_undegraded":"2024-04-02T11:33:44.087235+0000","last_fullsized":"2024-04-02T11:33:44.087235+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.618701+0000","last_change":"2024-04-02T11:32:04.403716+0000","last_active":"2024-04-02T11:33:44.618701+0000","last_peered":"2024-04-02T11:33:44.618701+0000","last_clean":"2024-04-02T11:33:44.618701+0000","last_became_active":"2024-04-02T11:32:04.390779+0000","last_became_peered":"2024-04-02T11:32:04.390779+0000","last_unstale":"2024-04-02T11:33:44.618701+0000","last_undegraded":"2024-04-02T11:33:44.618701+0000","last_fullsized":"2024-04-02T11:33:44.618701+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.8","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.743680+0000","last_change":"2024-04-02T11:33:41.322754+0000","last_active":"2024-04-02T11:33:44.743680+0000","last_peered":"2024-04-02T11:33:44.743680+0000","last_clean":"2024-04-02T11:33:44.743680+0000","last_became_active":"2024-04-02T11:33:41.311852+0000","last_became_peered":"2024-04-02T11:33:41.311852+0000","last_unstale":"2024-04-02T11:33:44.743680+0000","last_undegraded":"2024-04-02T11:33:44.743680+0000","last_fullsized":"2024-04-02T11:33:44.743680+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.749745+0000","last_change":"2024-04-02T11:32:04.441727+0000","last_active":"2024-04-02T11:33:44.749745+0000","last_peered":"2024-04-02T11:33:44.749745+0000","last_clean":"2024-04-02T11:33:44.749745+0000","last_became_active":"2024-04-02T11:32:04.409646+0000","last_became_peered":"2024-04-02T11:32:04.409646+0000","last_unstale":"2024-04-02T11:33:44.749745+0000","last_undegraded":"2024-04-02T11:33:44.749745+0000","last_fullsized":"2024-04-02T11:33:44.749745+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.9","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.773128+0000","last_change":"2024-04-02T11:33:41.293468+0000","last_active":"2024-04-02T11:33:44.773128+0000","last_peered":"2024-04-02T11:33:44.773128+0000","last_clean":"2024-04-02T11:33:44.773128+0000","last_became_active":"2024-04-02T11:33:41.267676+0000","last_became_peered":"2024-04-02T11:33:41.267676+0000","last_unstale":"2024-04-02T11:33:44.773128+0000","last_undegraded":"2024-04-02T11:33:44.773128+0000","last_fullsized":"2024-04-02T11:33:44.773128+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.509804+0000","last_change":"2024-04-02T11:32:04.404984+0000","last_active":"2024-04-02T11:33:44.509804+0000","last_peered":"2024-04-02T11:33:44.509804+0000","last_clean":"2024-04-02T11:33:44.509804+0000","last_became_active":"2024-04-02T11:32:04.387697+0000","last_became_peered":"2024-04-02T11:32:04.387697+0000","last_unstale":"2024-04-02T11:33:44.509804+0000","last_undegraded":"2024-04-02T11:33:44.509804+0000","last_fullsized":"2024-04-02T11:33:44.509804+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.755181+0000","last_change":"2024-04-02T11:33:41.296184+0000","last_active":"2024-04-02T11:33:44.755181+0000","last_peered":"2024-04-02T11:33:44.755181+0000","last_clean":"2024-04-02T11:33:44.755181+0000","last_became_active":"2024-04-02T11:33:41.271175+0000","last_became_peered":"2024-04-02T11:33:41.271175+0000","last_unstale":"2024-04-02T11:33:44.755181+0000","last_undegraded":"2024-04-02T11:33:44.755181+0000","last_fullsized":"2024-04-02T11:33:44.755181+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.732609+0000","last_change":"2024-04-02T11:32:04.423708+0000","last_active":"2024-04-02T11:33:44.732609+0000","last_peered":"2024-04-02T11:33:44.732609+0000","last_clean":"2024-04-02T11:33:44.732609+0000","last_became_active":"2024-04-02T11:32:04.401780+0000","last_became_peered":"2024-04-02T11:32:04.401780+0000","last_unstale":"2024-04-02T11:33:44.732609+0000","last_undegraded":"2024-04-02T11:33:44.732609+0000","last_fullsized":"2024-04-02T11:33:44.732609+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.742027+0000","last_change":"2024-04-02T11:33:41.274959+0000","last_active":"2024-04-02T11:33:44.742027+0000","last_peered":"2024-04-02T11:33:44.742027+0000","last_clean":"2024-04-02T11:33:44.742027+0000","last_became_active":"2024-04-02T11:33:41.254717+0000","last_became_peered":"2024-04-02T11:33:41.254717+0000","last_unstale":"2024-04-02T11:33:44.742027+0000","last_undegraded":"2024-04-02T11:33:44.742027+0000","last_fullsized":"2024-04-02T11:33:44.742027+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.728056+0000","last_change":"2024-04-02T11:32:04.457456+0000","last_active":"2024-04-02T11:33:44.728056+0000","last_peered":"2024-04-02T11:33:44.728056+0000","last_clean":"2024-04-02T11:33:44.728056+0000","last_became_active":"2024-04-02T11:32:04.436936+0000","last_became_peered":"2024-04-02T11:32:04.436936+0000","last_unstale":"2024-04-02T11:33:44.728056+0000","last_undegraded":"2024-04-02T11:33:44.728056+0000","last_fullsized":"2024-04-02T11:33:44.728056+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.579024+0000","last_change":"2024-04-02T11:33:41.259181+0000","last_active":"2024-04-02T11:33:44.579024+0000","last_peered":"2024-04-02T11:33:44.579024+0000","last_clean":"2024-04-02T11:33:44.579024+0000","last_became_active":"2024-04-02T11:33:41.249228+0000","last_became_peered":"2024-04-02T11:33:41.249228+0000","last_unstale":"2024-04-02T11:33:44.579024+0000","last_undegraded":"2024-04-02T11:33:44.579024+0000","last_fullsized":"2024-04-02T11:33:44.579024+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.503670+0000","last_change":"2024-04-02T11:32:04.429226+0000","last_active":"2024-04-02T11:33:44.503670+0000","last_peered":"2024-04-02T11:33:44.503670+0000","last_clean":"2024-04-02T11:33:44.503670+0000","last_became_active":"2024-04-02T11:32:04.407776+0000","last_became_peered":"2024-04-02T11:32:04.407776+0000","last_unstale":"2024-04-02T11:33:44.503670+0000","last_undegraded":"2024-04-02T11:33:44.503670+0000","last_fullsized":"2024-04-02T11:33:44.503670+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.480198+0000","last_change":"2024-04-02T11:33:41.329049+0000","last_active":"2024-04-02T11:33:44.480198+0000","last_peered":"2024-04-02T11:33:44.480198+0000","last_clean":"2024-04-02T11:33:44.480198+0000","last_became_active":"2024-04-02T11:33:41.323281+0000","last_became_peered":"2024-04-02T11:33:41.323281+0000","last_unstale":"2024-04-02T11:33:44.480198+0000","last_undegraded":"2024-04-02T11:33:44.480198+0000","last_fullsized":"2024-04-02T11:33:44.480198+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.494895+0000","last_change":"2024-04-02T11:32:04.558136+0000","last_active":"2024-04-02T11:33:44.494895+0000","last_peered":"2024-04-02T11:33:44.494895+0000","last_clean":"2024-04-02T11:33:44.494895+0000","last_became_active":"2024-04-02T11:32:04.551595+0000","last_became_peered":"2024-04-02T11:32:04.551595+0000","last_unstale":"2024-04-02T11:33:44.494895+0000","last_undegraded":"2024-04-02T11:33:44.494895+0000","last_fullsized":"2024-04-02T11:33:44.494895+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.12","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.478405+0000","last_change":"2024-04-02T11:33:41.281874+0000","last_active":"2024-04-02T11:33:44.478405+0000","last_peered":"2024-04-02T11:33:44.478405+0000","last_clean":"2024-04-02T11:33:44.478405+0000","last_became_active":"2024-04-02T11:33:41.274203+0000","last_became_peered":"2024-04-02T11:33:41.274203+0000","last_unstale":"2024-04-02T11:33:44.478405+0000","last_undegraded":"2024-04-02T11:33:44.478405+0000","last_fullsized":"2024-04-02T11:33:44.478405+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.485930+0000","last_change":"2024-04-02T11:32:04.417231+0000","last_active":"2024-04-02T11:33:44.485930+0000","last_peered":"2024-04-02T11:33:44.485930+0000","last_clean":"2024-04-02T11:33:44.485930+0000","last_became_active":"2024-04-02T11:32:04.396181+0000","last_became_peered":"2024-04-02T11:32:04.396181+0000","last_unstale":"2024-04-02T11:33:44.485930+0000","last_undegraded":"2024-04-02T11:33:44.485930+0000","last_fullsized":"2024-04-02T11:33:44.485930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.13","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.602064+0000","last_change":"2024-04-02T11:33:41.245713+0000","last_active":"2024-04-02T11:33:44.602064+0000","last_peered":"2024-04-02T11:33:44.602064+0000","last_clean":"2024-04-02T11:33:44.602064+0000","last_became_active":"2024-04-02T11:33:41.226115+0000","last_became_peered":"2024-04-02T11:33:41.226115+0000","last_unstale":"2024-04-02T11:33:44.602064+0000","last_undegraded":"2024-04-02T11:33:44.602064+0000","last_fullsized":"2024-04-02T11:33:44.602064+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.754146+0000","last_change":"2024-04-02T11:32:04.450092+0000","last_active":"2024-04-02T11:33:44.754146+0000","last_peered":"2024-04-02T11:33:44.754146+0000","last_clean":"2024-04-02T11:33:44.754146+0000","last_became_active":"2024-04-02T11:32:04.418023+0000","last_became_peered":"2024-04-02T11:32:04.418023+0000","last_unstale":"2024-04-02T11:33:44.754146+0000","last_undegraded":"2024-04-02T11:33:44.754146+0000","last_fullsized":"2024-04-02T11:33:44.754146+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.10","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.572668+0000","last_change":"2024-04-02T11:33:41.266477+0000","last_active":"2024-04-02T11:33:44.572668+0000","last_peered":"2024-04-02T11:33:44.572668+0000","last_clean":"2024-04-02T11:33:44.572668+0000","last_became_active":"2024-04-02T11:33:41.256091+0000","last_became_peered":"2024-04-02T11:33:41.256091+0000","last_unstale":"2024-04-02T11:33:44.572668+0000","last_undegraded":"2024-04-02T11:33:44.572668+0000","last_fullsized":"2024-04-02T11:33:44.572668+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.611376+0000","last_change":"2024-04-02T11:32:04.397771+0000","last_active":"2024-04-02T11:33:44.611376+0000","last_peered":"2024-04-02T11:33:44.611376+0000","last_clean":"2024-04-02T11:33:44.611376+0000","last_became_active":"2024-04-02T11:32:04.378859+0000","last_became_peered":"2024-04-02T11:32:04.378859+0000","last_unstale":"2024-04-02T11:33:44.611376+0000","last_undegraded":"2024-04-02T11:33:44.611376+0000","last_fullsized":"2024-04-02T11:33:44.611376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.11","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.758547+0000","last_change":"2024-04-02T11:33:41.263642+0000","last_active":"2024-04-02T11:33:44.758547+0000","last_peered":"2024-04-02T11:33:44.758547+0000","last_clean":"2024-04-02T11:33:44.758547+0000","last_became_active":"2024-04-02T11:33:41.250607+0000","last_became_peered":"2024-04-02T11:33:41.250607+0000","last_unstale":"2024-04-02T11:33:44.758547+0000","last_undegraded":"2024-04-02T11:33:44.758547+0000","last_fullsized":"2024-04-02T11:33:44.758547+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.073706+0000","last_change":"2024-04-02T11:32:04.463785+0000","last_active":"2024-04-02T11:33:44.073706+0000","last_peered":"2024-04-02T11:33:44.073706+0000","last_clean":"2024-04-02T11:33:44.073706+0000","last_became_active":"2024-04-02T11:32:04.457870+0000","last_became_peered":"2024-04-02T11:32:04.457870+0000","last_unstale":"2024-04-02T11:33:44.073706+0000","last_undegraded":"2024-04-02T11:33:44.073706+0000","last_fullsized":"2024-04-02T11:33:44.073706+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.16","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.605469+0000","last_change":"2024-04-02T11:33:41.241536+0000","last_active":"2024-04-02T11:33:44.605469+0000","last_peered":"2024-04-02T11:33:44.605469+0000","last_clean":"2024-04-02T11:33:44.605469+0000","last_became_active":"2024-04-02T11:33:41.232699+0000","last_became_peered":"2024-04-02T11:33:41.232699+0000","last_unstale":"2024-04-02T11:33:44.605469+0000","last_undegraded":"2024-04-02T11:33:44.605469+0000","last_fullsized":"2024-04-02T11:33:44.605469+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.071157+0000","last_change":"2024-04-02T11:32:04.384861+0000","last_active":"2024-04-02T11:33:44.071157+0000","last_peered":"2024-04-02T11:33:44.071157+0000","last_clean":"2024-04-02T11:33:44.071157+0000","last_became_active":"2024-04-02T11:32:04.368304+0000","last_became_peered":"2024-04-02T11:32:04.368304+0000","last_unstale":"2024-04-02T11:33:44.071157+0000","last_undegraded":"2024-04-02T11:33:44.071157+0000","last_fullsized":"2024-04-02T11:33:44.071157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.17","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.590092+0000","last_change":"2024-04-02T11:33:41.281940+0000","last_active":"2024-04-02T11:33:44.590092+0000","last_peered":"2024-04-02T11:33:44.590092+0000","last_clean":"2024-04-02T11:33:44.590092+0000","last_became_active":"2024-04-02T11:33:41.275074+0000","last_became_peered":"2024-04-02T11:33:41.275074+0000","last_unstale":"2024-04-02T11:33:44.590092+0000","last_undegraded":"2024-04-02T11:33:44.590092+0000","last_fullsized":"2024-04-02T11:33:44.590092+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":138,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.091094+0000","last_change":"2024-04-02T11:32:04.377416+0000","last_active":"2024-04-02T11:33:44.091094+0000","last_peered":"2024-04-02T11:33:44.091094+0000","last_clean":"2024-04-02T11:33:44.091094+0000","last_became_active":"2024-04-02T11:32:04.364464+0000","last_became_peered":"2024-04-02T11:32:04.364464+0000","last_unstale":"2024-04-02T11:33:44.091094+0000","last_undegraded":"2024-04-02T11:33:44.091094+0000","last_fullsized":"2024-04-02T11:33:44.091094+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.14","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.076833+0000","last_change":"2024-04-02T11:33:41.311816+0000","last_active":"2024-04-02T11:33:44.076833+0000","last_peered":"2024-04-02T11:33:44.076833+0000","last_clean":"2024-04-02T11:33:44.076833+0000","last_became_active":"2024-04-02T11:33:41.290381+0000","last_became_peered":"2024-04-02T11:33:41.290381+0000","last_unstale":"2024-04-02T11:33:44.076833+0000","last_undegraded":"2024-04-02T11:33:44.076833+0000","last_fullsized":"2024-04-02T11:33:44.076833+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.599260+0000","last_change":"2024-04-02T11:32:04.407271+0000","last_active":"2024-04-02T11:33:44.599260+0000","last_peered":"2024-04-02T11:33:44.599260+0000","last_clean":"2024-04-02T11:33:44.599260+0000","last_became_active":"2024-04-02T11:32:04.386636+0000","last_became_peered":"2024-04-02T11:32:04.386636+0000","last_unstale":"2024-04-02T11:33:44.599260+0000","last_undegraded":"2024-04-02T11:33:44.599260+0000","last_fullsized":"2024-04-02T11:33:44.599260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.15","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.085724+0000","last_change":"2024-04-02T11:33:41.299177+0000","last_active":"2024-04-02T11:33:44.085724+0000","last_peered":"2024-04-02T11:33:44.085724+0000","last_clean":"2024-04-02T11:33:44.085724+0000","last_became_active":"2024-04-02T11:33:41.271590+0000","last_became_peered":"2024-04-02T11:33:41.271590+0000","last_unstale":"2024-04-02T11:33:44.085724+0000","last_undegraded":"2024-04-02T11:33:44.085724+0000","last_fullsized":"2024-04-02T11:33:44.085724+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.703445+0000","last_change":"2024-04-02T11:32:04.421594+0000","last_active":"2024-04-02T11:33:44.703445+0000","last_peered":"2024-04-02T11:33:44.703445+0000","last_clean":"2024-04-02T11:33:44.703445+0000","last_became_active":"2024-04-02T11:32:04.404970+0000","last_became_peered":"2024-04-02T11:32:04.404970+0000","last_unstale":"2024-04-02T11:33:44.703445+0000","last_undegraded":"2024-04-02T11:33:44.703445+0000","last_fullsized":"2024-04-02T11:33:44.703445+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1a","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113166+0000","last_change":"2024-04-02T11:33:41.294167+0000","last_active":"2024-04-02T11:33:44.113166+0000","last_peered":"2024-04-02T11:33:44.113166+0000","last_clean":"2024-04-02T11:33:44.113166+0000","last_became_active":"2024-04-02T11:33:41.278749+0000","last_became_peered":"2024-04-02T11:33:41.278749+0000","last_unstale":"2024-04-02T11:33:44.113166+0000","last_undegraded":"2024-04-02T11:33:44.113166+0000","last_fullsized":"2024-04-02T11:33:44.113166+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.753274+0000","last_change":"2024-04-02T11:32:04.439591+0000","last_active":"2024-04-02T11:33:44.753274+0000","last_peered":"2024-04-02T11:33:44.753274+0000","last_clean":"2024-04-02T11:33:44.753274+0000","last_became_active":"2024-04-02T11:32:04.414684+0000","last_became_peered":"2024-04-02T11:32:04.414684+0000","last_unstale":"2024-04-02T11:33:44.753274+0000","last_undegraded":"2024-04-02T11:33:44.753274+0000","last_fullsized":"2024-04-02T11:33:44.753274+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1b","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.739455+0000","last_change":"2024-04-02T11:33:41.277360+0000","last_active":"2024-04-02T11:33:44.739455+0000","last_peered":"2024-04-02T11:33:44.739455+0000","last_clean":"2024-04-02T11:33:44.739455+0000","last_became_active":"2024-04-02T11:33:41.258789+0000","last_became_peered":"2024-04-02T11:33:41.258789+0000","last_unstale":"2024-04-02T11:33:44.739455+0000","last_undegraded":"2024-04-02T11:33:44.739455+0000","last_fullsized":"2024-04-02T11:33:44.739455+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.510847+0000","last_change":"2024-04-02T11:32:04.419077+0000","last_active":"2024-04-02T11:33:44.510847+0000","last_peered":"2024-04-02T11:33:44.510847+0000","last_clean":"2024-04-02T11:33:44.510847+0000","last_became_active":"2024-04-02T11:32:04.392023+0000","last_became_peered":"2024-04-02T11:32:04.392023+0000","last_unstale":"2024-04-02T11:33:44.510847+0000","last_undegraded":"2024-04-02T11:33:44.510847+0000","last_fullsized":"2024-04-02T11:33:44.510847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.18","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.501017+0000","last_change":"2024-04-02T11:33:41.379485+0000","last_active":"2024-04-02T11:33:44.501017+0000","last_peered":"2024-04-02T11:33:44.501017+0000","last_clean":"2024-04-02T11:33:44.501017+0000","last_became_active":"2024-04-02T11:33:41.374309+0000","last_became_peered":"2024-04-02T11:33:41.374309+0000","last_unstale":"2024-04-02T11:33:44.501017+0000","last_undegraded":"2024-04-02T11:33:44.501017+0000","last_fullsized":"2024-04-02T11:33:44.501017+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.766807+0000","last_change":"2024-04-02T11:32:04.464793+0000","last_active":"2024-04-02T11:33:44.766807+0000","last_peered":"2024-04-02T11:33:44.766807+0000","last_clean":"2024-04-02T11:33:44.766807+0000","last_became_active":"2024-04-02T11:32:04.445170+0000","last_became_peered":"2024-04-02T11:32:04.445170+0000","last_unstale":"2024-04-02T11:33:44.766807+0000","last_undegraded":"2024-04-02T11:33:44.766807+0000","last_fullsized":"2024-04-02T11:33:44.766807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.19","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.493963+0000","last_change":"2024-04-02T11:33:41.353471+0000","last_active":"2024-04-02T11:33:44.493963+0000","last_peered":"2024-04-02T11:33:44.493963+0000","last_clean":"2024-04-02T11:33:44.493963+0000","last_became_active":"2024-04-02T11:33:41.347559+0000","last_became_peered":"2024-04-02T11:33:41.347559+0000","last_unstale":"2024-04-02T11:33:44.493963+0000","last_undegraded":"2024-04-02T11:33:44.493963+0000","last_fullsized":"2024-04-02T11:33:44.493963+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.492318+0000","last_change":"2024-04-02T11:32:04.474632+0000","last_active":"2024-04-02T11:33:44.492318+0000","last_peered":"2024-04-02T11:33:44.492318+0000","last_clean":"2024-04-02T11:33:44.492318+0000","last_became_active":"2024-04-02T11:32:04.467593+0000","last_became_peered":"2024-04-02T11:32:04.467593+0000","last_unstale":"2024-04-02T11:33:44.492318+0000","last_undegraded":"2024-04-02T11:33:44.492318+0000","last_fullsized":"2024-04-02T11:33:44.492318+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1e","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.765959+0000","last_change":"2024-04-02T11:33:41.303867+0000","last_active":"2024-04-02T11:33:44.765959+0000","last_peered":"2024-04-02T11:33:44.765959+0000","last_clean":"2024-04-02T11:33:44.765959+0000","last_became_active":"2024-04-02T11:33:41.282720+0000","last_became_peered":"2024-04-02T11:33:41.282720+0000","last_unstale":"2024-04-02T11:33:44.765959+0000","last_undegraded":"2024-04-02T11:33:44.765959+0000","last_fullsized":"2024-04-02T11:33:44.765959+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.489009+0000","last_change":"2024-04-02T11:32:04.415345+0000","last_active":"2024-04-02T11:33:44.489009+0000","last_peered":"2024-04-02T11:33:44.489009+0000","last_clean":"2024-04-02T11:33:44.489009+0000","last_became_active":"2024-04-02T11:32:04.400375+0000","last_became_peered":"2024-04-02T11:32:04.400375+0000","last_unstale":"2024-04-02T11:33:44.489009+0000","last_undegraded":"2024-04-02T11:33:44.489009+0000","last_fullsized":"2024-04-02T11:33:44.489009+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1f","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.777113+0000","last_change":"2024-04-02T11:33:41.334657+0000","last_active":"2024-04-02T11:33:44.777113+0000","last_peered":"2024-04-02T11:33:44.777113+0000","last_clean":"2024-04-02T11:33:44.777113+0000","last_became_active":"2024-04-02T11:33:41.325633+0000","last_became_peered":"2024-04-02T11:33:41.325633+0000","last_unstale":"2024-04-02T11:33:44.777113+0000","last_undegraded":"2024-04-02T11:33:44.777113+0000","last_fullsized":"2024-04-02T11:33:44.777113+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.471747+0000","last_change":"2024-04-02T11:32:04.461967+0000","last_active":"2024-04-02T11:33:44.471747+0000","last_peered":"2024-04-02T11:33:44.471747+0000","last_clean":"2024-04-02T11:33:44.471747+0000","last_became_active":"2024-04-02T11:32:04.456013+0000","last_became_peered":"2024-04-02T11:32:04.456013+0000","last_unstale":"2024-04-02T11:33:44.471747+0000","last_undegraded":"2024-04-02T11:33:44.471747+0000","last_fullsized":"2024-04-02T11:33:44.471747+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1c","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.506496+0000","last_change":"2024-04-02T11:33:41.285447+0000","last_active":"2024-04-02T11:33:44.506496+0000","last_peered":"2024-04-02T11:33:44.506496+0000","last_clean":"2024-04-02T11:33:44.506496+0000","last_became_active":"2024-04-02T11:33:41.270749+0000","last_became_peered":"2024-04-02T11:33:41.270749+0000","last_unstale":"2024-04-02T11:33:44.506496+0000","last_undegraded":"2024-04-02T11:33:44.506496+0000","last_fullsized":"2024-04-02T11:33:44.506496+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":151,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.074943+0000","last_change":"2024-04-02T11:32:04.431094+0000","last_active":"2024-04-02T11:33:44.074943+0000","last_peered":"2024-04-02T11:33:44.074943+0000","last_clean":"2024-04-02T11:33:44.074943+0000","last_became_active":"2024-04-02T11:32:04.423058+0000","last_became_peered":"2024-04-02T11:32:04.423058+0000","last_unstale":"2024-04-02T11:33:44.074943+0000","last_undegraded":"2024-04-02T11:33:44.074943+0000","last_fullsized":"2024-04-02T11:33:44.074943+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.1d","version":"0'0","reported_seq":14,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.106087+0000","last_change":"2024-04-02T11:33:41.274565+0000","last_active":"2024-04-02T11:33:44.106087+0000","last_peered":"2024-04-02T11:33:44.106087+0000","last_clean":"2024-04-02T11:33:44.106087+0000","last_became_active":"2024-04-02T11:33:41.258583+0000","last_became_peered":"2024-04-02T11:33:41.258583+0000","last_unstale":"2024-04-02T11:33:44.106087+0000","last_undegraded":"2024-04-02T11:33:44.106087+0000","last_fullsized":"2024-04-02T11:33:44.106087+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:33:40.041882+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":144,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.593632+0000","last_change":"2024-04-02T11:32:04.401482+0000","last_active":"2024-04-02T11:33:44.593632+0000","last_peered":"2024-04-02T11:33:44.593632+0000","last_clean":"2024-04-02T11:33:44.593632+0000","last_became_active":"2024-04-02T11:32:04.383344+0000","last_became_peered":"2024-04-02T11:32:04.383344+0000","last_unstale":"2024-04-02T11:33:44.593632+0000","last_undegraded":"2024-04-02T11:33:44.593632+0000","last_fullsized":"2024-04-02T11:33:44.593632+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:32:03.164536+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.069239+0000","last_change":"2024-04-02T11:26:56.195483+0000","last_active":"2024-04-02T11:33:44.069239+0000","last_peered":"2024-04-02T11:33:44.069239+0000","last_clean":"2024-04-02T11:33:44.069239+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T11:33:44.069239+0000","last_undegraded":"2024-04-02T11:33:44.069239+0000","last_fullsized":"2024-04-02T11:33:44.069239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.627702+0000","last_change":"2024-04-02T11:26:56.169654+0000","last_active":"2024-04-02T11:33:44.627702+0000","last_peered":"2024-04-02T11:33:44.627702+0000","last_clean":"2024-04-02T11:33:44.627702+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T11:33:44.627702+0000","last_undegraded":"2024-04-02T11:33:44.627702+0000","last_fullsized":"2024-04-02T11:33:44.627702+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":741,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.722011+0000","last_change":"2024-04-02T11:26:56.180485+0000","last_active":"2024-04-02T11:33:44.722011+0000","last_peered":"2024-04-02T11:33:44.722011+0000","last_clean":"2024-04-02T11:33:44.722011+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T11:33:44.722011+0000","last_undegraded":"2024-04-02T11:33:44.722011+0000","last_fullsized":"2024-04-02T11:33:44.722011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.623600+0000","last_change":"2024-04-02T11:26:56.164391+0000","last_active":"2024-04-02T11:33:44.623600+0000","last_peered":"2024-04-02T11:33:44.623600+0000","last_clean":"2024-04-02T11:33:44.623600+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T11:33:44.623600+0000","last_undegraded":"2024-04-02T11:33:44.623600+0000","last_fullsized":"2024-04-02T11:33:44.623600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.473736+0000","last_change":"2024-04-02T11:26:56.178086+0000","last_active":"2024-04-02T11:33:44.473736+0000","last_peered":"2024-04-02T11:33:44.473736+0000","last_clean":"2024-04-02T11:33:44.473736+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T11:33:44.473736+0000","last_undegraded":"2024-04-02T11:33:44.473736+0000","last_fullsized":"2024-04-02T11:33:44.473736+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.110855+0000","last_change":"2024-04-02T11:26:56.166505+0000","last_active":"2024-04-02T11:33:44.110855+0000","last_peered":"2024-04-02T11:33:44.110855+0000","last_clean":"2024-04-02T11:33:44.110855+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T11:33:44.110855+0000","last_undegraded":"2024-04-02T11:33:44.110855+0000","last_fullsized":"2024-04-02T11:33:44.110855+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.504908+0000","last_change":"2024-04-02T11:26:56.178139+0000","last_active":"2024-04-02T11:33:44.504908+0000","last_peered":"2024-04-02T11:33:44.504908+0000","last_clean":"2024-04-02T11:33:44.504908+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T11:33:44.504908+0000","last_undegraded":"2024-04-02T11:33:44.504908+0000","last_fullsized":"2024-04-02T11:33:44.504908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":733,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.113298+0000","last_change":"2024-04-02T11:26:56.179257+0000","last_active":"2024-04-02T11:33:44.113298+0000","last_peered":"2024-04-02T11:33:44.113298+0000","last_clean":"2024-04-02T11:33:44.113298+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T11:33:44.113298+0000","last_undegraded":"2024-04-02T11:33:44.113298+0000","last_fullsized":"2024-04-02T11:33:44.113298+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:55.099376+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":877,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T11:33:44.734120+0000","last_change":"2024-04-02T11:26:52.134441+0000","last_active":"2024-04-02T11:33:44.734120+0000","last_peered":"2024-04-02T11:33:44.734120+0000","last_clean":"2024-04-02T11:33:44.734120+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T11:33:44.734120+0000","last_undegraded":"2024-04-02T11:33:44.734120+0000","last_fullsized":"2024-04-02T11:33:44.734120+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T11:26:51.083297+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":86,"num_pg":32,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":4},{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654716524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":31104,"kb_avail":104824672,"statfs":{"total":107374182400,"available":107340464128,"internally_reserved":0,"allocated":1859584,"data_stored":1037844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31850496},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654716525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829396,"statfs":{"total":107374182400,"available":107345301504,"internally_reserved":0,"allocated":1282048,"data_stored":451769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654716525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":54576,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104803024,"statfs":{"total":107374182400,"available":107318296576,"internally_reserved":0,"allocated":1286144,"data_stored":456057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654716524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":55140,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104802460,"statfs":{"total":107374182400,"available":107317719040,"internally_reserved":0,"allocated":1863680,"data_stored":1037972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:28:04.740 INFO:tasks.ceph:pgid 86.2 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.740 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.740 INFO:tasks.ceph:pgid 86.3 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.740 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.740 INFO:tasks.ceph:pgid 86.0 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 86.1 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 86.6 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.741 INFO:tasks.ceph:pgid 86.7 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 86.4 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 86.5 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.742 INFO:tasks.ceph:pgid 86.a last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 86.b last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 86.8 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.743 INFO:tasks.ceph:pgid 86.9 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 86.e last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 86.f last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.744 INFO:tasks.ceph:pgid 86.c last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 86.d last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 86.12 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.745 INFO:tasks.ceph:pgid 86.13 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.746 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.746 INFO:tasks.ceph:pgid 86.10 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.746 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.746 INFO:tasks.ceph:pgid 86.11 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.746 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 86.16 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 86.17 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 86.14 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.747 INFO:tasks.ceph:pgid 86.15 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 86.1a last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 86.1b last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 86.18 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.748 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 86.19 last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 86.1e last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 86.1f last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.749 INFO:tasks.ceph:pgid 86.1c last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 86.1d last_scrub_stamp 2024-04-02T11:33:40.041882+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=33, tm_sec=40, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-04-02T11:32:03.164536+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=32, tm_sec=3, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.750 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-02T11:26:55.099376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=55, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-02T11:26:51.083297+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=11, tm_min=26, tm_sec=51, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=14, tm_min=28, tm_sec=1, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T14:28:04.751 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-02T14:28:24.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T14:28:24.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.872+0000 7fd31c19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:24.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.877+0000 7fd31c19d640 1 Processor -- start 2024-04-02T14:28:24.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.877+0000 7fd31c19d640 1 -- start start 2024-04-02T14:28:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.878+0000 7fd31c19d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd31409c820 0x7fd314131840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.878+0000 7fd31c19d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31406d300 con 0x7fd314131d80 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.878+0000 7fd31c19d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7fd31409cce0 con 0x7fd31409c820 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319f39640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd31409c820 0x7fd314131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319738640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd314131d80 0x7fd31406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44628/0 (socket says 172.21.15.133:44628) 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319738640 1 -- 172.21.15.133:0/3142929295 learned_addr learned my addr 172.21.15.133:0/3142929295 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319f39640 1 -- 172.21.15.133:0/3142929295 >> v1:172.21.15.133:6789/0 conn(0x7fd314131d80 legacy=0x7fd31406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319f39640 1 -- 172.21.15.133:0/3142929295 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31409c380 con 0x7fd31409c820 2024-04-02T14:28:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd318f37640 1 -- 172.21.15.133:0/3142929295 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148122328 0 0) 0x7fd31406d300 con 0x7fd314131d80 2024-04-02T14:28:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd319f39640 1 --2- 172.21.15.133:0/3142929295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd31409c820 0x7fd314131840 crc :-1 s=READY pgs=12347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a7667bc1d57b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd318f37640 1 -- 172.21.15.133:0/3142929295 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd31000e040 con 0x7fd31409c820 2024-04-02T14:28:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd318f37640 1 -- 172.21.15.133:0/3142929295 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd310007670 con 0x7fd31409c820 2024-04-02T14:28:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.879+0000 7fd318f37640 1 -- 172.21.15.133:0/3142929295 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd310007910 con 0x7fd31409c820 2024-04-02T14:28:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 -- 172.21.15.133:0/3142929295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd31409c820 msgr2=0x7fd314131840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3142929295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd31409c820 0x7fd314131840 crc :-1 s=READY pgs=12347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 -- 172.21.15.133:0/3142929295 shutdown_connections 2024-04-02T14:28:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3142929295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd31409c820 0x7fd314131840 unknown :-1 s=CLOSED pgs=12347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 -- 172.21.15.133:0/3142929295 >> 172.21.15.133:0/3142929295 conn(0x7fd3140a0790 msgr2=0x7fd3140a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 -- 172.21.15.133:0/3142929295 shutdown_connections 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 1 -- 172.21.15.133:0/3142929295 wait complete. 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.880+0000 7fd31c19d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.882+0000 7fd31c19d640 1 Processor -- start 2024-04-02T14:28:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.882+0000 7fd31c19d640 1 -- start start 2024-04-02T14:28:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd31c19d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd31c19d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd31411fd40 con 0x7fd314131d80 2024-04-02T14:28:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd319f39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd319f39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54382/0 (socket says 172.21.15.133:54382) 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd319f39640 1 -- 172.21.15.133:0/3472027377 learned_addr learned my addr 172.21.15.133:0/3472027377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd319f39640 1 -- 172.21.15.133:0/3472027377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141c2410 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd319f39640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 crc :-1 s=READY pgs=12348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd31000e040 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd310013aa0 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3141c2720 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd310013d40 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.883+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31411ec30 con 0x7fd314131d80 2024-04-02T14:28:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.884+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7fd310011040 con 0x7fd314131d80 2024-04-02T14:28:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.884+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3141c2910 con 0x7fd314131d80 2024-04-02T14:28:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.884+0000 7fd30a7fc640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 0x7fd2f803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.885+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7fd310049e70 con 0x7fd314131d80 2024-04-02T14:28:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.885+0000 7fd319738640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 0x7fd2f803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.886+0000 7fd319738640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 0x7fd2f803ae90 crc :-1 s=READY pgs=3811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:24.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:24.890+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7fd310016e40 con 0x7fd314131d80 2024-04-02T14:28:25.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.065+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 --> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd314121a80 con 0x7fd2f8038980 2024-04-02T14:28:25.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.071+0000 7fd30a7fc640 1 -- 172.21.15.133:0/3472027377 <== mgr.4101 v2:172.21.15.138:6800/1817191150 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+187059 (crc 0 0 0) 0x7fd314121a80 con 0x7fd2f8038980 2024-04-02T14:28:25.074 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-02T14:28:25.074 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-02T14:28:25.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 msgr2=0x7fd2f803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 0x7fd2f803ae90 crc :-1 s=READY pgs=3811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 msgr2=0x7fd3141c2ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 crc :-1 s=READY pgs=12348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 shutdown_connections 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7fd2f8038980 0x7fd2f803ae90 unknown :-1 s=CLOSED pgs=3811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 --2- 172.21.15.133:0/3472027377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd314131d80 0x7fd3141c2ac0 unknown :-1 s=CLOSED pgs=12348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 >> 172.21.15.133:0/3472027377 conn(0x7fd3140a0790 msgr2=0x7fd31412f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 shutdown_connections 2024-04-02T14:28:25.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.073+0000 7fd31c19d640 1 -- 172.21.15.133:0/3472027377 wait complete. 2024-04-02T14:28:25.086 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":5725,"stamp":"2024-04-02T14:28:23.862113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594469,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":145,"num_write_kb":1905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":170848,"kb_used_data":6144,"kb_used_omap":0,"kb_used_meta":164672,"kb_avail":419259552,"statfs":{"total":429496729600,"available":429321781248,"internally_reserved":0,"allocated":6291456,"data_stored":2983642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":168624128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003671"},"pg_stats":[{"pgid":"86.2","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.486855+0000","last_change":"2024-04-02T14:28:03.534855+0000","last_active":"2024-04-02T14:28:03.486855+0000","last_peered":"2024-04-02T14:28:03.486855+0000","last_clean":"2024-04-02T14:28:03.486855+0000","last_became_active":"2024-04-02T11:33:41.342906+0000","last_became_peered":"2024-04-02T11:33:41.342906+0000","last_unstale":"2024-04-02T14:28:03.486855+0000","last_undegraded":"2024-04-02T14:28:03.486855+0000","last_fullsized":"2024-04-02T14:28:03.486855+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.534700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.534700+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1f","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.480998+0000","last_change":"2024-04-02T14:28:03.532435+0000","last_active":"2024-04-02T14:28:03.480998+0000","last_peered":"2024-04-02T14:28:03.480998+0000","last_clean":"2024-04-02T14:28:03.480998+0000","last_became_active":"2024-04-02T11:32:04.372115+0000","last_became_peered":"2024-04-02T11:32:04.372115+0000","last_unstale":"2024-04-02T14:28:03.480998+0000","last_undegraded":"2024-04-02T14:28:03.480998+0000","last_fullsized":"2024-04-02T14:28:03.480998+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.532257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.532257+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.3","version":"285'1","reported_seq":18,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.488991+0000","last_change":"2024-04-02T14:28:03.531315+0000","last_active":"2024-04-02T14:28:03.488991+0000","last_peered":"2024-04-02T14:28:03.488991+0000","last_clean":"2024-04-02T14:28:03.488991+0000","last_became_active":"2024-04-02T11:33:41.286642+0000","last_became_peered":"2024-04-02T11:33:41.286642+0000","last_unstale":"2024-04-02T14:28:03.488991+0000","last_undegraded":"2024-04-02T14:28:03.488991+0000","last_fullsized":"2024-04-02T14:28:03.488991+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'1","last_scrub_stamp":"2024-04-02T14:28:03.531117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.531117+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.028503+0000","last_change":"2024-04-02T14:28:03.062692+0000","last_active":"2024-04-02T14:28:03.028503+0000","last_peered":"2024-04-02T14:28:03.028503+0000","last_clean":"2024-04-02T14:28:03.028503+0000","last_became_active":"2024-04-02T11:32:04.424122+0000","last_became_peered":"2024-04-02T11:32:04.424122+0000","last_unstale":"2024-04-02T14:28:03.028503+0000","last_undegraded":"2024-04-02T14:28:03.028503+0000","last_fullsized":"2024-04-02T14:28:03.028503+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.062517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.062517+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.0","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.487590+0000","last_change":"2024-04-02T14:28:03.533104+0000","last_active":"2024-04-02T14:28:03.487590+0000","last_peered":"2024-04-02T14:28:03.487590+0000","last_clean":"2024-04-02T14:28:03.487590+0000","last_became_active":"2024-04-02T11:33:41.268384+0000","last_became_peered":"2024-04-02T11:33:41.268384+0000","last_unstale":"2024-04-02T14:28:03.487590+0000","last_undegraded":"2024-04-02T14:28:03.487590+0000","last_fullsized":"2024-04-02T14:28:03.487590+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.532922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.532922+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":146,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.950893+0000","last_change":"2024-04-02T14:28:03.996456+0000","last_active":"2024-04-02T14:28:03.950893+0000","last_peered":"2024-04-02T14:28:03.950893+0000","last_clean":"2024-04-02T14:28:03.950893+0000","last_became_active":"2024-04-02T11:32:04.394162+0000","last_became_peered":"2024-04-02T11:32:04.394162+0000","last_unstale":"2024-04-02T14:28:03.950893+0000","last_undegraded":"2024-04-02T14:28:03.950893+0000","last_fullsized":"2024-04-02T14:28:03.950893+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.996280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.996280+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.1","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.430255+0000","last_change":"2024-04-02T14:28:04.496691+0000","last_active":"2024-04-02T14:28:04.430255+0000","last_peered":"2024-04-02T14:28:04.430255+0000","last_clean":"2024-04-02T14:28:04.430255+0000","last_became_active":"2024-04-02T11:33:41.317736+0000","last_became_peered":"2024-04-02T11:33:41.317736+0000","last_unstale":"2024-04-02T14:28:04.430255+0000","last_undegraded":"2024-04-02T14:28:04.430255+0000","last_fullsized":"2024-04-02T14:28:04.430255+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.496535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.496535+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.425495+0000","last_change":"2024-04-02T14:28:04.488641+0000","last_active":"2024-04-02T14:28:04.425495+0000","last_peered":"2024-04-02T14:28:04.425495+0000","last_clean":"2024-04-02T14:28:04.425495+0000","last_became_active":"2024-04-02T11:32:04.430454+0000","last_became_peered":"2024-04-02T11:32:04.430454+0000","last_unstale":"2024-04-02T14:28:04.425495+0000","last_undegraded":"2024-04-02T14:28:04.425495+0000","last_fullsized":"2024-04-02T14:28:04.425495+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.488479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.488479+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.6","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.954228+0000","last_change":"2024-04-02T14:28:03.980710+0000","last_active":"2024-04-02T14:28:03.954228+0000","last_peered":"2024-04-02T14:28:03.954228+0000","last_clean":"2024-04-02T14:28:03.954228+0000","last_became_active":"2024-04-02T11:33:41.236833+0000","last_became_peered":"2024-04-02T11:33:41.236833+0000","last_unstale":"2024-04-02T14:28:03.954228+0000","last_undegraded":"2024-04-02T14:28:03.954228+0000","last_fullsized":"2024-04-02T14:28:03.954228+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.980518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.980518+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.480367+0000","last_change":"2024-04-02T14:28:03.518130+0000","last_active":"2024-04-02T14:28:03.480367+0000","last_peered":"2024-04-02T14:28:03.480367+0000","last_clean":"2024-04-02T14:28:03.480367+0000","last_became_active":"2024-04-02T11:32:04.392132+0000","last_became_peered":"2024-04-02T11:32:04.392132+0000","last_unstale":"2024-04-02T14:28:03.480367+0000","last_undegraded":"2024-04-02T14:28:03.480367+0000","last_fullsized":"2024-04-02T14:28:03.480367+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.517957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.517957+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.7","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.436872+0000","last_change":"2024-04-02T14:28:04.474361+0000","last_active":"2024-04-02T14:28:04.436872+0000","last_peered":"2024-04-02T14:28:04.436872+0000","last_clean":"2024-04-02T14:28:04.436872+0000","last_became_active":"2024-04-02T11:33:41.288766+0000","last_became_peered":"2024-04-02T11:33:41.288766+0000","last_unstale":"2024-04-02T14:28:04.436872+0000","last_undegraded":"2024-04-02T14:28:04.436872+0000","last_fullsized":"2024-04-02T14:28:04.436872+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.474169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.474169+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.025192+0000","last_change":"2024-04-02T14:28:03.048835+0000","last_active":"2024-04-02T14:28:03.025192+0000","last_peered":"2024-04-02T14:28:03.025192+0000","last_clean":"2024-04-02T14:28:03.025192+0000","last_became_active":"2024-04-02T11:32:04.435642+0000","last_became_peered":"2024-04-02T11:32:04.435642+0000","last_unstale":"2024-04-02T14:28:03.025192+0000","last_undegraded":"2024-04-02T14:28:03.025192+0000","last_fullsized":"2024-04-02T14:28:03.025192+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.048672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.048672+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.4","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.434949+0000","last_change":"2024-04-02T14:28:04.498000+0000","last_active":"2024-04-02T14:28:04.434949+0000","last_peered":"2024-04-02T14:28:04.434949+0000","last_clean":"2024-04-02T14:28:04.434949+0000","last_became_active":"2024-04-02T11:33:41.230755+0000","last_became_peered":"2024-04-02T11:33:41.230755+0000","last_unstale":"2024-04-02T14:28:04.434949+0000","last_undegraded":"2024-04-02T14:28:04.434949+0000","last_fullsized":"2024-04-02T14:28:04.434949+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.497840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.497840+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.020419+0000","last_change":"2024-04-02T14:28:03.072509+0000","last_active":"2024-04-02T14:28:03.020419+0000","last_peered":"2024-04-02T14:28:03.020419+0000","last_clean":"2024-04-02T14:28:03.020419+0000","last_became_active":"2024-04-02T11:32:04.523227+0000","last_became_peered":"2024-04-02T11:32:04.523227+0000","last_unstale":"2024-04-02T14:28:03.020419+0000","last_undegraded":"2024-04-02T14:28:03.020419+0000","last_fullsized":"2024-04-02T14:28:03.020419+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.072234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.072234+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.5","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.955373+0000","last_change":"2024-04-02T14:28:03.997860+0000","last_active":"2024-04-02T14:28:03.955373+0000","last_peered":"2024-04-02T14:28:03.955373+0000","last_clean":"2024-04-02T14:28:03.955373+0000","last_became_active":"2024-04-02T11:33:41.229522+0000","last_became_peered":"2024-04-02T11:33:41.229522+0000","last_unstale":"2024-04-02T14:28:03.955373+0000","last_undegraded":"2024-04-02T14:28:03.955373+0000","last_fullsized":"2024-04-02T14:28:03.955373+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.997691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.997691+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.015582+0000","last_change":"2024-04-02T14:28:03.066651+0000","last_active":"2024-04-02T14:28:03.015582+0000","last_peered":"2024-04-02T14:28:03.015582+0000","last_clean":"2024-04-02T14:28:03.015582+0000","last_became_active":"2024-04-02T11:32:04.490360+0000","last_became_peered":"2024-04-02T11:32:04.490360+0000","last_unstale":"2024-04-02T14:28:03.015582+0000","last_undegraded":"2024-04-02T14:28:03.015582+0000","last_fullsized":"2024-04-02T14:28:03.015582+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.066485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.066485+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.a","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.956823+0000","last_change":"2024-04-02T14:28:03.983545+0000","last_active":"2024-04-02T14:28:03.956823+0000","last_peered":"2024-04-02T14:28:03.956823+0000","last_clean":"2024-04-02T14:28:03.956823+0000","last_became_active":"2024-04-02T11:33:41.301597+0000","last_became_peered":"2024-04-02T11:33:41.301597+0000","last_unstale":"2024-04-02T14:28:03.956823+0000","last_undegraded":"2024-04-02T14:28:03.956823+0000","last_fullsized":"2024-04-02T14:28:03.956823+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.983346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.983346+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.422762+0000","last_change":"2024-04-02T14:28:04.495603+0000","last_active":"2024-04-02T14:28:04.422762+0000","last_peered":"2024-04-02T14:28:04.422762+0000","last_clean":"2024-04-02T14:28:04.422762+0000","last_became_active":"2024-04-02T11:32:04.427391+0000","last_became_peered":"2024-04-02T11:32:04.427391+0000","last_unstale":"2024-04-02T14:28:04.422762+0000","last_undegraded":"2024-04-02T14:28:04.422762+0000","last_fullsized":"2024-04-02T14:28:04.422762+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.495445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.495445+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.b","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.492143+0000","last_change":"2024-04-02T14:28:03.520711+0000","last_active":"2024-04-02T14:28:03.492143+0000","last_peered":"2024-04-02T14:28:03.492143+0000","last_clean":"2024-04-02T14:28:03.492143+0000","last_became_active":"2024-04-02T11:33:41.264133+0000","last_became_peered":"2024-04-02T11:33:41.264133+0000","last_unstale":"2024-04-02T14:28:03.492143+0000","last_undegraded":"2024-04-02T14:28:03.492143+0000","last_fullsized":"2024-04-02T14:28:03.492143+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.520534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.520534+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":146,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.948889+0000","last_change":"2024-04-02T14:28:03.993040+0000","last_active":"2024-04-02T14:28:03.948889+0000","last_peered":"2024-04-02T14:28:03.948889+0000","last_clean":"2024-04-02T14:28:03.948889+0000","last_became_active":"2024-04-02T11:32:04.390779+0000","last_became_peered":"2024-04-02T11:32:04.390779+0000","last_unstale":"2024-04-02T14:28:03.948889+0000","last_undegraded":"2024-04-02T14:28:03.948889+0000","last_fullsized":"2024-04-02T14:28:03.948889+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.992851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.992851+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.8","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.443597+0000","last_change":"2024-04-02T14:28:04.480152+0000","last_active":"2024-04-02T14:28:04.443597+0000","last_peered":"2024-04-02T14:28:04.443597+0000","last_clean":"2024-04-02T14:28:04.443597+0000","last_became_active":"2024-04-02T11:33:41.311852+0000","last_became_peered":"2024-04-02T11:33:41.311852+0000","last_unstale":"2024-04-02T14:28:04.443597+0000","last_undegraded":"2024-04-02T14:28:04.443597+0000","last_fullsized":"2024-04-02T14:28:04.443597+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.479985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.479985+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.417260+0000","last_change":"2024-04-02T14:28:04.489549+0000","last_active":"2024-04-02T14:28:04.417260+0000","last_peered":"2024-04-02T14:28:04.417260+0000","last_clean":"2024-04-02T14:28:04.417260+0000","last_became_active":"2024-04-02T11:32:04.409646+0000","last_became_peered":"2024-04-02T11:32:04.409646+0000","last_unstale":"2024-04-02T14:28:04.417260+0000","last_undegraded":"2024-04-02T14:28:04.417260+0000","last_fullsized":"2024-04-02T14:28:04.417260+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.489391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.489391+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.9","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.447456+0000","last_change":"2024-04-02T14:28:04.484838+0000","last_active":"2024-04-02T14:28:04.447456+0000","last_peered":"2024-04-02T14:28:04.447456+0000","last_clean":"2024-04-02T14:28:04.447456+0000","last_became_active":"2024-04-02T11:33:41.267676+0000","last_became_peered":"2024-04-02T11:33:41.267676+0000","last_unstale":"2024-04-02T14:28:04.447456+0000","last_undegraded":"2024-04-02T14:28:04.447456+0000","last_fullsized":"2024-04-02T14:28:04.447456+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.484680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.484680+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.011914+0000","last_change":"2024-04-02T14:28:03.063663+0000","last_active":"2024-04-02T14:28:03.011914+0000","last_peered":"2024-04-02T14:28:03.011914+0000","last_clean":"2024-04-02T14:28:03.011914+0000","last_became_active":"2024-04-02T11:32:04.387697+0000","last_became_peered":"2024-04-02T11:32:04.387697+0000","last_unstale":"2024-04-02T14:28:03.011914+0000","last_undegraded":"2024-04-02T14:28:03.011914+0000","last_fullsized":"2024-04-02T14:28:03.011914+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.063488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.063488+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.e","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.450414+0000","last_change":"2024-04-02T14:28:04.500504+0000","last_active":"2024-04-02T14:28:04.450414+0000","last_peered":"2024-04-02T14:28:04.450414+0000","last_clean":"2024-04-02T14:28:04.450414+0000","last_became_active":"2024-04-02T11:33:41.271175+0000","last_became_peered":"2024-04-02T11:33:41.271175+0000","last_unstale":"2024-04-02T14:28:04.450414+0000","last_undegraded":"2024-04-02T14:28:04.450414+0000","last_fullsized":"2024-04-02T14:28:04.450414+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.500348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.500348+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.421158+0000","last_change":"2024-04-02T14:28:04.493014+0000","last_active":"2024-04-02T14:28:04.421158+0000","last_peered":"2024-04-02T14:28:04.421158+0000","last_clean":"2024-04-02T14:28:04.421158+0000","last_became_active":"2024-04-02T11:32:04.401780+0000","last_became_peered":"2024-04-02T11:32:04.401780+0000","last_unstale":"2024-04-02T14:28:04.421158+0000","last_undegraded":"2024-04-02T14:28:04.421158+0000","last_fullsized":"2024-04-02T14:28:04.421158+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.492846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.492846+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.f","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.452141+0000","last_change":"2024-04-02T14:28:04.501303+0000","last_active":"2024-04-02T14:28:04.452141+0000","last_peered":"2024-04-02T14:28:04.452141+0000","last_clean":"2024-04-02T14:28:04.452141+0000","last_became_active":"2024-04-02T11:33:41.254717+0000","last_became_peered":"2024-04-02T11:33:41.254717+0000","last_unstale":"2024-04-02T14:28:04.452141+0000","last_undegraded":"2024-04-02T14:28:04.452141+0000","last_fullsized":"2024-04-02T14:28:04.452141+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.501129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.501129+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.415304+0000","last_change":"2024-04-02T14:28:04.486029+0000","last_active":"2024-04-02T14:28:04.415304+0000","last_peered":"2024-04-02T14:28:04.415304+0000","last_clean":"2024-04-02T14:28:04.415304+0000","last_became_active":"2024-04-02T11:32:04.436936+0000","last_became_peered":"2024-04-02T11:32:04.436936+0000","last_unstale":"2024-04-02T14:28:04.415304+0000","last_undegraded":"2024-04-02T14:28:04.415304+0000","last_fullsized":"2024-04-02T14:28:04.415304+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.485864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.485864+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.c","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.958511+0000","last_change":"2024-04-02T14:28:03.985137+0000","last_active":"2024-04-02T14:28:03.958511+0000","last_peered":"2024-04-02T14:28:03.958511+0000","last_clean":"2024-04-02T14:28:03.958511+0000","last_became_active":"2024-04-02T11:33:41.249228+0000","last_became_peered":"2024-04-02T11:33:41.249228+0000","last_unstale":"2024-04-02T14:28:03.958511+0000","last_undegraded":"2024-04-02T14:28:03.958511+0000","last_fullsized":"2024-04-02T14:28:03.958511+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.984926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.984926+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.010907+0000","last_change":"2024-04-02T14:28:03.060540+0000","last_active":"2024-04-02T14:28:03.010907+0000","last_peered":"2024-04-02T14:28:03.010907+0000","last_clean":"2024-04-02T14:28:03.010907+0000","last_became_active":"2024-04-02T11:32:04.407776+0000","last_became_peered":"2024-04-02T11:32:04.407776+0000","last_unstale":"2024-04-02T14:28:03.010907+0000","last_undegraded":"2024-04-02T14:28:03.010907+0000","last_fullsized":"2024-04-02T14:28:03.010907+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.060383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.060383+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.d","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.029345+0000","last_change":"2024-04-02T14:28:03.059879+0000","last_active":"2024-04-02T14:28:03.029345+0000","last_peered":"2024-04-02T14:28:03.029345+0000","last_clean":"2024-04-02T14:28:03.029345+0000","last_became_active":"2024-04-02T11:33:41.323281+0000","last_became_peered":"2024-04-02T11:33:41.323281+0000","last_unstale":"2024-04-02T14:28:03.029345+0000","last_undegraded":"2024-04-02T14:28:03.029345+0000","last_fullsized":"2024-04-02T14:28:03.029345+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.059723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.059723+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.005859+0000","last_change":"2024-04-02T14:28:03.055473+0000","last_active":"2024-04-02T14:28:03.005859+0000","last_peered":"2024-04-02T14:28:03.005859+0000","last_clean":"2024-04-02T14:28:03.005859+0000","last_became_active":"2024-04-02T11:32:04.551595+0000","last_became_peered":"2024-04-02T11:32:04.551595+0000","last_unstale":"2024-04-02T14:28:03.005859+0000","last_undegraded":"2024-04-02T14:28:03.005859+0000","last_fullsized":"2024-04-02T14:28:03.005859+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.055209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.055209+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.12","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.032434+0000","last_change":"2024-04-02T14:28:03.068389+0000","last_active":"2024-04-02T14:28:03.032434+0000","last_peered":"2024-04-02T14:28:03.032434+0000","last_clean":"2024-04-02T14:28:03.032434+0000","last_became_active":"2024-04-02T11:33:41.274203+0000","last_became_peered":"2024-04-02T11:33:41.274203+0000","last_unstale":"2024-04-02T14:28:03.032434+0000","last_undegraded":"2024-04-02T14:28:03.032434+0000","last_fullsized":"2024-04-02T14:28:03.032434+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.068218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.068218+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.004841+0000","last_change":"2024-04-02T14:28:03.058725+0000","last_active":"2024-04-02T14:28:03.004841+0000","last_peered":"2024-04-02T14:28:03.004841+0000","last_clean":"2024-04-02T14:28:03.004841+0000","last_became_active":"2024-04-02T11:32:04.396181+0000","last_became_peered":"2024-04-02T11:32:04.396181+0000","last_unstale":"2024-04-02T14:28:03.004841+0000","last_undegraded":"2024-04-02T14:28:03.004841+0000","last_fullsized":"2024-04-02T14:28:03.004841+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.058566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.058566+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.13","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.963893+0000","last_change":"2024-04-02T14:28:03.986704+0000","last_active":"2024-04-02T14:28:03.963893+0000","last_peered":"2024-04-02T14:28:03.963893+0000","last_clean":"2024-04-02T14:28:03.963893+0000","last_became_active":"2024-04-02T11:33:41.226115+0000","last_became_peered":"2024-04-02T11:33:41.226115+0000","last_unstale":"2024-04-02T14:28:03.963893+0000","last_undegraded":"2024-04-02T14:28:03.963893+0000","last_fullsized":"2024-04-02T14:28:03.963893+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.986518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.986518+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.413588+0000","last_change":"2024-04-02T14:28:04.483804+0000","last_active":"2024-04-02T14:28:04.413588+0000","last_peered":"2024-04-02T14:28:04.413588+0000","last_clean":"2024-04-02T14:28:04.413588+0000","last_became_active":"2024-04-02T11:32:04.418023+0000","last_became_peered":"2024-04-02T11:32:04.418023+0000","last_unstale":"2024-04-02T14:28:04.413588+0000","last_undegraded":"2024-04-02T14:28:04.413588+0000","last_fullsized":"2024-04-02T14:28:04.413588+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.483645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.483645+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.10","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.962722+0000","last_change":"2024-04-02T14:28:03.988006+0000","last_active":"2024-04-02T14:28:03.962722+0000","last_peered":"2024-04-02T14:28:03.962722+0000","last_clean":"2024-04-02T14:28:03.962722+0000","last_became_active":"2024-04-02T11:33:41.256091+0000","last_became_peered":"2024-04-02T11:33:41.256091+0000","last_unstale":"2024-04-02T14:28:03.962722+0000","last_undegraded":"2024-04-02T14:28:03.962722+0000","last_fullsized":"2024-04-02T14:28:03.962722+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.987785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.987785+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":146,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.947274+0000","last_change":"2024-04-02T14:28:03.978359+0000","last_active":"2024-04-02T14:28:03.947274+0000","last_peered":"2024-04-02T14:28:03.947274+0000","last_clean":"2024-04-02T14:28:03.947274+0000","last_became_active":"2024-04-02T11:32:04.378859+0000","last_became_peered":"2024-04-02T11:32:04.378859+0000","last_unstale":"2024-04-02T14:28:03.947274+0000","last_undegraded":"2024-04-02T14:28:03.947274+0000","last_fullsized":"2024-04-02T14:28:03.947274+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.978147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.978147+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.11","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.456415+0000","last_change":"2024-04-02T14:28:04.490633+0000","last_active":"2024-04-02T14:28:04.456415+0000","last_peered":"2024-04-02T14:28:04.456415+0000","last_clean":"2024-04-02T14:28:04.456415+0000","last_became_active":"2024-04-02T11:33:41.250607+0000","last_became_peered":"2024-04-02T11:33:41.250607+0000","last_unstale":"2024-04-02T14:28:04.456415+0000","last_undegraded":"2024-04-02T14:28:04.456415+0000","last_fullsized":"2024-04-02T14:28:04.456415+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.490455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.490455+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.476553+0000","last_change":"2024-04-02T14:28:03.511329+0000","last_active":"2024-04-02T14:28:03.476553+0000","last_peered":"2024-04-02T14:28:03.476553+0000","last_clean":"2024-04-02T14:28:03.476553+0000","last_became_active":"2024-04-02T11:32:04.457870+0000","last_became_peered":"2024-04-02T11:32:04.457870+0000","last_unstale":"2024-04-02T14:28:03.476553+0000","last_undegraded":"2024-04-02T14:28:03.476553+0000","last_fullsized":"2024-04-02T14:28:03.476553+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.511151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.511151+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.16","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.970584+0000","last_change":"2024-04-02T14:28:03.990187+0000","last_active":"2024-04-02T14:28:03.970584+0000","last_peered":"2024-04-02T14:28:03.970584+0000","last_clean":"2024-04-02T14:28:03.970584+0000","last_became_active":"2024-04-02T11:33:41.232699+0000","last_became_peered":"2024-04-02T11:33:41.232699+0000","last_unstale":"2024-04-02T14:28:03.970584+0000","last_undegraded":"2024-04-02T14:28:03.970584+0000","last_fullsized":"2024-04-02T14:28:03.970584+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.990008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.990008+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.474939+0000","last_change":"2024-04-02T14:28:03.517158+0000","last_active":"2024-04-02T14:28:03.474939+0000","last_peered":"2024-04-02T14:28:03.474939+0000","last_clean":"2024-04-02T14:28:03.474939+0000","last_became_active":"2024-04-02T11:32:04.368304+0000","last_became_peered":"2024-04-02T11:32:04.368304+0000","last_unstale":"2024-04-02T14:28:03.474939+0000","last_undegraded":"2024-04-02T14:28:03.474939+0000","last_fullsized":"2024-04-02T14:28:03.474939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.516973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.516973+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.17","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.973955+0000","last_change":"2024-04-02T14:28:03.991025+0000","last_active":"2024-04-02T14:28:03.973955+0000","last_peered":"2024-04-02T14:28:03.973955+0000","last_clean":"2024-04-02T14:28:03.973955+0000","last_became_active":"2024-04-02T11:33:41.275074+0000","last_became_peered":"2024-04-02T11:33:41.275074+0000","last_unstale":"2024-04-02T14:28:03.973955+0000","last_undegraded":"2024-04-02T14:28:03.973955+0000","last_fullsized":"2024-04-02T14:28:03.973955+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.990806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.990806+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":140,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.475938+0000","last_change":"2024-04-02T14:28:03.512967+0000","last_active":"2024-04-02T14:28:03.475938+0000","last_peered":"2024-04-02T14:28:03.475938+0000","last_clean":"2024-04-02T14:28:03.475938+0000","last_became_active":"2024-04-02T11:32:04.364464+0000","last_became_peered":"2024-04-02T11:32:04.364464+0000","last_unstale":"2024-04-02T14:28:03.475938+0000","last_undegraded":"2024-04-02T14:28:03.475938+0000","last_fullsized":"2024-04-02T14:28:03.475938+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.512684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.512684+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.14","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.494921+0000","last_change":"2024-04-02T14:28:03.521410+0000","last_active":"2024-04-02T14:28:03.494921+0000","last_peered":"2024-04-02T14:28:03.494921+0000","last_clean":"2024-04-02T14:28:03.494921+0000","last_became_active":"2024-04-02T11:33:41.290381+0000","last_became_peered":"2024-04-02T11:33:41.290381+0000","last_unstale":"2024-04-02T14:28:03.494921+0000","last_undegraded":"2024-04-02T14:28:03.494921+0000","last_fullsized":"2024-04-02T14:28:03.494921+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.521252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.521252+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":146,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.949671+0000","last_change":"2024-04-02T14:28:03.994795+0000","last_active":"2024-04-02T14:28:03.949671+0000","last_peered":"2024-04-02T14:28:03.949671+0000","last_clean":"2024-04-02T14:28:03.949671+0000","last_became_active":"2024-04-02T11:32:04.386636+0000","last_became_peered":"2024-04-02T11:32:04.386636+0000","last_unstale":"2024-04-02T14:28:03.949671+0000","last_undegraded":"2024-04-02T14:28:03.949671+0000","last_fullsized":"2024-04-02T14:28:03.949671+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.994626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.994626+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"86.15","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.496209+0000","last_change":"2024-04-02T14:28:03.536243+0000","last_active":"2024-04-02T14:28:03.496209+0000","last_peered":"2024-04-02T14:28:03.496209+0000","last_clean":"2024-04-02T14:28:03.496209+0000","last_became_active":"2024-04-02T11:33:41.271590+0000","last_became_peered":"2024-04-02T11:33:41.271590+0000","last_unstale":"2024-04-02T14:28:03.496209+0000","last_undegraded":"2024-04-02T14:28:03.496209+0000","last_fullsized":"2024-04-02T14:28:03.496209+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.536085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.536085+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.410705+0000","last_change":"2024-04-02T14:28:04.481056+0000","last_active":"2024-04-02T14:28:04.410705+0000","last_peered":"2024-04-02T14:28:04.410705+0000","last_clean":"2024-04-02T14:28:04.410705+0000","last_became_active":"2024-04-02T11:32:04.404970+0000","last_became_peered":"2024-04-02T11:32:04.404970+0000","last_unstale":"2024-04-02T14:28:04.410705+0000","last_undegraded":"2024-04-02T14:28:04.410705+0000","last_fullsized":"2024-04-02T14:28:04.410705+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.480864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.480864+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1a","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.504360+0000","last_change":"2024-04-02T14:28:03.525446+0000","last_active":"2024-04-02T14:28:03.504360+0000","last_peered":"2024-04-02T14:28:03.504360+0000","last_clean":"2024-04-02T14:28:03.504360+0000","last_became_active":"2024-04-02T11:33:41.278749+0000","last_became_peered":"2024-04-02T11:33:41.278749+0000","last_unstale":"2024-04-02T14:28:03.504360+0000","last_undegraded":"2024-04-02T14:28:03.504360+0000","last_fullsized":"2024-04-02T14:28:03.504360+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.525277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.525277+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.409293+0000","last_change":"2024-04-02T14:28:04.473436+0000","last_active":"2024-04-02T14:28:04.409293+0000","last_peered":"2024-04-02T14:28:04.409293+0000","last_clean":"2024-04-02T14:28:04.409293+0000","last_became_active":"2024-04-02T11:32:04.414684+0000","last_became_peered":"2024-04-02T11:32:04.414684+0000","last_unstale":"2024-04-02T14:28:04.409293+0000","last_undegraded":"2024-04-02T14:28:04.409293+0000","last_fullsized":"2024-04-02T14:28:04.409293+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.473249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.473249+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.1b","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.455200+0000","last_change":"2024-04-02T14:28:04.503295+0000","last_active":"2024-04-02T14:28:04.455200+0000","last_peered":"2024-04-02T14:28:04.455200+0000","last_clean":"2024-04-02T14:28:04.455200+0000","last_became_active":"2024-04-02T11:33:41.258789+0000","last_became_peered":"2024-04-02T11:33:41.258789+0000","last_unstale":"2024-04-02T14:28:04.455200+0000","last_undegraded":"2024-04-02T14:28:04.455200+0000","last_fullsized":"2024-04-02T14:28:04.455200+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.503131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.503131+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.003930+0000","last_change":"2024-04-02T14:28:03.054202+0000","last_active":"2024-04-02T14:28:03.003930+0000","last_peered":"2024-04-02T14:28:03.003930+0000","last_clean":"2024-04-02T14:28:03.003930+0000","last_became_active":"2024-04-02T11:32:04.392023+0000","last_became_peered":"2024-04-02T11:32:04.392023+0000","last_unstale":"2024-04-02T14:28:03.003930+0000","last_undegraded":"2024-04-02T14:28:03.003930+0000","last_fullsized":"2024-04-02T14:28:03.003930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.054010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.054010+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.18","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.025200+0000","last_change":"2024-04-02T14:28:03.071015+0000","last_active":"2024-04-02T14:28:03.025200+0000","last_peered":"2024-04-02T14:28:03.025200+0000","last_clean":"2024-04-02T14:28:03.025200+0000","last_became_active":"2024-04-02T11:33:41.374309+0000","last_became_peered":"2024-04-02T11:33:41.374309+0000","last_unstale":"2024-04-02T14:28:03.025200+0000","last_undegraded":"2024-04-02T14:28:03.025200+0000","last_fullsized":"2024-04-02T14:28:03.025200+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.070843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.070843+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.407128+0000","last_change":"2024-04-02T14:28:04.476406+0000","last_active":"2024-04-02T14:28:04.407128+0000","last_peered":"2024-04-02T14:28:04.407128+0000","last_clean":"2024-04-02T14:28:04.407128+0000","last_became_active":"2024-04-02T11:32:04.445170+0000","last_became_peered":"2024-04-02T11:32:04.445170+0000","last_unstale":"2024-04-02T14:28:04.407128+0000","last_undegraded":"2024-04-02T14:28:04.407128+0000","last_fullsized":"2024-04-02T14:28:04.407128+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.476245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.476245+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"86.19","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.033419+0000","last_change":"2024-04-02T14:28:03.067706+0000","last_active":"2024-04-02T14:28:03.033419+0000","last_peered":"2024-04-02T14:28:03.033419+0000","last_clean":"2024-04-02T14:28:03.033419+0000","last_became_active":"2024-04-02T11:33:41.347559+0000","last_became_peered":"2024-04-02T11:33:41.347559+0000","last_unstale":"2024-04-02T14:28:03.033419+0000","last_undegraded":"2024-04-02T14:28:03.033419+0000","last_fullsized":"2024-04-02T14:28:03.033419+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.067541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.067541+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.002563+0000","last_change":"2024-04-02T14:28:03.050786+0000","last_active":"2024-04-02T14:28:03.002563+0000","last_peered":"2024-04-02T14:28:03.002563+0000","last_clean":"2024-04-02T14:28:03.002563+0000","last_became_active":"2024-04-02T11:32:04.467593+0000","last_became_peered":"2024-04-02T11:32:04.467593+0000","last_unstale":"2024-04-02T14:28:03.002563+0000","last_undegraded":"2024-04-02T14:28:03.002563+0000","last_fullsized":"2024-04-02T14:28:03.002563+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.050621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.050621+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1e","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.460894+0000","last_change":"2024-04-02T14:28:04.503942+0000","last_active":"2024-04-02T14:28:04.460894+0000","last_peered":"2024-04-02T14:28:04.460894+0000","last_clean":"2024-04-02T14:28:04.460894+0000","last_became_active":"2024-04-02T11:33:41.282720+0000","last_became_peered":"2024-04-02T11:33:41.282720+0000","last_unstale":"2024-04-02T14:28:04.460894+0000","last_undegraded":"2024-04-02T14:28:04.460894+0000","last_fullsized":"2024-04-02T14:28:04.460894+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.503775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.503775+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.001466+0000","last_change":"2024-04-02T14:28:03.051573+0000","last_active":"2024-04-02T14:28:03.001466+0000","last_peered":"2024-04-02T14:28:03.001466+0000","last_clean":"2024-04-02T14:28:03.001466+0000","last_became_active":"2024-04-02T11:32:04.400375+0000","last_became_peered":"2024-04-02T11:32:04.400375+0000","last_unstale":"2024-04-02T14:28:03.001466+0000","last_undegraded":"2024-04-02T14:28:03.001466+0000","last_fullsized":"2024-04-02T14:28:03.001466+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.051420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.051420+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1f","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.463468+0000","last_change":"2024-04-02T14:28:04.494566+0000","last_active":"2024-04-02T14:28:04.463468+0000","last_peered":"2024-04-02T14:28:04.463468+0000","last_clean":"2024-04-02T14:28:04.463468+0000","last_became_active":"2024-04-02T11:33:41.325633+0000","last_became_peered":"2024-04-02T11:33:41.325633+0000","last_unstale":"2024-04-02T14:28:04.463468+0000","last_undegraded":"2024-04-02T14:28:04.463468+0000","last_fullsized":"2024-04-02T14:28:04.463468+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.494399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.494399+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":142,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:02.999888+0000","last_change":"2024-04-02T14:28:03.046614+0000","last_active":"2024-04-02T14:28:02.999888+0000","last_peered":"2024-04-02T14:28:02.999888+0000","last_clean":"2024-04-02T14:28:02.999888+0000","last_became_active":"2024-04-02T11:32:04.456013+0000","last_became_peered":"2024-04-02T11:32:04.456013+0000","last_unstale":"2024-04-02T14:28:02.999888+0000","last_undegraded":"2024-04-02T14:28:02.999888+0000","last_fullsized":"2024-04-02T14:28:02.999888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.046451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.046451+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"86.1c","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.035972+0000","last_change":"2024-04-02T14:28:03.071410+0000","last_active":"2024-04-02T14:28:03.035972+0000","last_peered":"2024-04-02T14:28:03.035972+0000","last_clean":"2024-04-02T14:28:03.035972+0000","last_became_active":"2024-04-02T11:33:41.270749+0000","last_became_peered":"2024-04-02T11:33:41.270749+0000","last_unstale":"2024-04-02T14:28:03.035972+0000","last_undegraded":"2024-04-02T14:28:03.035972+0000","last_fullsized":"2024-04-02T14:28:03.035972+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.071241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.071241+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"237'7","reported_seq":153,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.473650+0000","last_change":"2024-04-02T14:28:03.528724+0000","last_active":"2024-04-02T14:28:03.473650+0000","last_peered":"2024-04-02T14:28:03.473650+0000","last_clean":"2024-04-02T14:28:03.473650+0000","last_became_active":"2024-04-02T11:32:04.423058+0000","last_became_peered":"2024-04-02T11:32:04.423058+0000","last_unstale":"2024-04-02T14:28:03.473650+0000","last_undegraded":"2024-04-02T14:28:03.473650+0000","last_fullsized":"2024-04-02T14:28:03.473650+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'7","last_scrub_stamp":"2024-04-02T14:28:03.528560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.528560+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"86.1d","version":"0'0","reported_seq":16,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.506129+0000","last_change":"2024-04-02T14:28:03.524695+0000","last_active":"2024-04-02T14:28:03.506129+0000","last_peered":"2024-04-02T14:28:03.506129+0000","last_clean":"2024-04-02T14:28:03.506129+0000","last_became_active":"2024-04-02T11:33:41.258583+0000","last_became_peered":"2024-04-02T11:33:41.258583+0000","last_unstale":"2024-04-02T14:28:03.506129+0000","last_undegraded":"2024-04-02T14:28:03.506129+0000","last_fullsized":"2024-04-02T14:28:03.506129+0000","mapping_epoch":284,"log_start":"0'0","ondisk_log_start":"0'0","created":284,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.524530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.524530+0000","last_clean_scrub_stamp":"2024-04-02T11:33:40.041882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":146,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.943250+0000","last_change":"2024-04-02T14:28:03.975927+0000","last_active":"2024-04-02T14:28:03.943250+0000","last_peered":"2024-04-02T14:28:03.943250+0000","last_clean":"2024-04-02T14:28:03.943250+0000","last_became_active":"2024-04-02T11:32:04.383344+0000","last_became_peered":"2024-04-02T11:32:04.383344+0000","last_unstale":"2024-04-02T14:28:03.943250+0000","last_undegraded":"2024-04-02T14:28:03.943250+0000","last_fullsized":"2024-04-02T14:28:03.943250+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.975715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.975715+0000","last_clean_scrub_stamp":"2024-04-02T11:32:03.164536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.471463+0000","last_change":"2024-04-02T14:28:03.498766+0000","last_active":"2024-04-02T14:28:03.471463+0000","last_peered":"2024-04-02T14:28:03.471463+0000","last_clean":"2024-04-02T14:28:03.471463+0000","last_became_active":"2024-04-02T11:26:56.189358+0000","last_became_peered":"2024-04-02T11:26:56.189358+0000","last_unstale":"2024-04-02T14:28:03.471463+0000","last_undegraded":"2024-04-02T14:28:03.471463+0000","last_fullsized":"2024-04-02T14:28:03.471463+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.498562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.498562+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":737,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.943411+0000","last_change":"2024-04-02T14:28:03.954540+0000","last_active":"2024-04-02T14:28:03.943411+0000","last_peered":"2024-04-02T14:28:03.943411+0000","last_clean":"2024-04-02T14:28:03.943411+0000","last_became_active":"2024-04-02T11:26:56.164306+0000","last_became_peered":"2024-04-02T11:26:56.164306+0000","last_unstale":"2024-04-02T14:28:03.943411+0000","last_undegraded":"2024-04-02T14:28:03.943411+0000","last_fullsized":"2024-04-02T14:28:03.943411+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.954313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.954313+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":743,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.406805+0000","last_change":"2024-04-02T14:28:04.453806+0000","last_active":"2024-04-02T14:28:04.406805+0000","last_peered":"2024-04-02T14:28:04.406805+0000","last_clean":"2024-04-02T14:28:04.406805+0000","last_became_active":"2024-04-02T11:26:56.175300+0000","last_became_peered":"2024-04-02T11:26:56.175300+0000","last_unstale":"2024-04-02T14:28:04.406805+0000","last_undegraded":"2024-04-02T14:28:04.406805+0000","last_fullsized":"2024-04-02T14:28:04.406805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:04.453485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.453485+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":737,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.940573+0000","last_change":"2024-04-02T14:28:03.989090+0000","last_active":"2024-04-02T14:28:03.940573+0000","last_peered":"2024-04-02T14:28:03.940573+0000","last_clean":"2024-04-02T14:28:03.940573+0000","last_became_active":"2024-04-02T11:26:56.158694+0000","last_became_peered":"2024-04-02T11:26:56.158694+0000","last_unstale":"2024-04-02T14:28:03.940573+0000","last_undegraded":"2024-04-02T14:28:03.940573+0000","last_fullsized":"2024-04-02T14:28:03.940573+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.988897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.988897+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":745,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:02.999401+0000","last_change":"2024-04-02T14:28:03.045123+0000","last_active":"2024-04-02T14:28:02.999401+0000","last_peered":"2024-04-02T14:28:02.999401+0000","last_clean":"2024-04-02T14:28:02.999401+0000","last_became_active":"2024-04-02T11:26:56.173527+0000","last_became_peered":"2024-04-02T11:26:56.173527+0000","last_unstale":"2024-04-02T14:28:02.999401+0000","last_undegraded":"2024-04-02T14:28:02.999401+0000","last_fullsized":"2024-04-02T14:28:02.999401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.044913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.044913+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.470487+0000","last_change":"2024-04-02T14:28:03.526831+0000","last_active":"2024-04-02T14:28:03.470487+0000","last_peered":"2024-04-02T14:28:03.470487+0000","last_clean":"2024-04-02T14:28:03.470487+0000","last_became_active":"2024-04-02T11:26:56.161381+0000","last_became_peered":"2024-04-02T11:26:56.161381+0000","last_unstale":"2024-04-02T14:28:03.470487+0000","last_undegraded":"2024-04-02T14:28:03.470487+0000","last_fullsized":"2024-04-02T14:28:03.470487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.526671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.526671+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":745,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:02.999501+0000","last_change":"2024-04-02T14:28:03.044609+0000","last_active":"2024-04-02T14:28:02.999501+0000","last_peered":"2024-04-02T14:28:02.999501+0000","last_clean":"2024-04-02T14:28:02.999501+0000","last_became_active":"2024-04-02T11:26:56.173363+0000","last_became_peered":"2024-04-02T11:26:56.173363+0000","last_unstale":"2024-04-02T14:28:02.999501+0000","last_undegraded":"2024-04-02T14:28:02.999501+0000","last_fullsized":"2024-04-02T14:28:02.999501+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.044435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.044435+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":735,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:03.470660+0000","last_change":"2024-04-02T14:28:03.479523+0000","last_active":"2024-04-02T14:28:03.470660+0000","last_peered":"2024-04-02T14:28:03.470660+0000","last_clean":"2024-04-02T14:28:03.470660+0000","last_became_active":"2024-04-02T11:26:56.172344+0000","last_became_peered":"2024-04-02T11:26:56.172344+0000","last_unstale":"2024-04-02T14:28:03.470660+0000","last_undegraded":"2024-04-02T14:28:03.470660+0000","last_fullsized":"2024-04-02T14:28:03.470660+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T14:28:03.479348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:03.479348+0000","last_clean_scrub_stamp":"2024-04-02T11:26:55.099376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":879,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-04-02T14:28:04.404329+0000","last_change":"2024-04-02T14:28:04.478695+0000","last_active":"2024-04-02T14:28:04.404329+0000","last_peered":"2024-04-02T14:28:04.404329+0000","last_clean":"2024-04-02T14:28:04.404329+0000","last_became_active":"2024-04-02T11:26:52.129777+0000","last_became_peered":"2024-04-02T11:26:52.129777+0000","last_unstale":"2024-04-02T14:28:04.404329+0000","last_undegraded":"2024-04-02T14:28:04.404329+0000","last_fullsized":"2024-04-02T14:28:04.404329+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'97","last_scrub_stamp":"2024-04-02T14:28:04.478503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T14:28:04.478503+0000","last_clean_scrub_stamp":"2024-04-02T11:26:51.083297+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":86,"num_pg":32,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":4},{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654716544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":31104,"kb_avail":104824672,"statfs":{"total":107374182400,"available":107340464128,"internally_reserved":0,"allocated":1859584,"data_stored":1037844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31850496},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654716545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829396,"statfs":{"total":107374182400,"available":107345301504,"internally_reserved":0,"allocated":1282048,"data_stored":451769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654716545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":54576,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104803024,"statfs":{"total":107374182400,"available":107318296576,"internally_reserved":0,"allocated":1286144,"data_stored":456057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654716544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":55140,"kb_used_data":1820,"kb_used_omap":0,"kb_used_meta":53312,"kb_avail":104802460,"statfs":{"total":107374182400,"available":107317719040,"internally_reserved":0,"allocated":1863680,"data_stored":1037972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54591488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":86,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T14:28:25.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-02T14:28:25.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.194+0000 7f5dbba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:25.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.201+0000 7f5dbba6d640 1 Processor -- start 2024-04-02T14:28:25.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.201+0000 7f5dbba6d640 1 -- start start 2024-04-02T14:28:25.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5dbba6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5db409edf0 0x7f5db409f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:25.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5dbba6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40a0070 con 0x7f5db409f740 2024-04-02T14:28:25.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5dbba6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40a03f0 con 0x7f5db409edf0 2024-04-02T14:28:25.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9809640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5db409edf0 0x7f5db409f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:25.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9008640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5db409f740 0x7f5db4131320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44632/0 (socket says 172.21.15.133:44632) 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9008640 1 -- 172.21.15.133:0/2896654700 learned_addr learned my addr 172.21.15.133:0/2896654700 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9809640 1 -- 172.21.15.133:0/2896654700 >> v1:172.21.15.133:6789/0 conn(0x7f5db409f740 legacy=0x7f5db4131320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9809640 1 -- 172.21.15.133:0/2896654700 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db406e8a0 con 0x7f5db409edf0 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.202+0000 7f5db9809640 1 --2- 172.21.15.133:0/2896654700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db409f200 crc :-1 s=READY pgs=12350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3597812984e44cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5db8807640 1 -- 172.21.15.133:0/2896654700 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5db000e040 con 0x7f5db409edf0 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5db8807640 1 -- 172.21.15.133:0/2896654700 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5db0007670 con 0x7f5db409edf0 2024-04-02T14:28:25.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5db8807640 1 -- 172.21.15.133:0/2896654700 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5db0007910 con 0x7f5db409edf0 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 -- 172.21.15.133:0/2896654700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 msgr2=0x7f5db409f200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/2896654700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db409f200 crc :-1 s=READY pgs=12350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 -- 172.21.15.133:0/2896654700 shutdown_connections 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/2896654700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db409f200 unknown :-1 s=CLOSED pgs=12350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 -- 172.21.15.133:0/2896654700 >> 172.21.15.133:0/2896654700 conn(0x7f5db40a3830 msgr2=0x7f5db40a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 -- 172.21.15.133:0/2896654700 shutdown_connections 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.203+0000 7f5dbba6d640 1 -- 172.21.15.133:0/2896654700 wait complete. 2024-04-02T14:28:25.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.204+0000 7f5dbba6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-04-02T14:28:25.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.205+0000 7f5dbba6d640 1 Processor -- start 2024-04-02T14:28:25.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.205+0000 7f5dbba6d640 1 -- start start 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5dbba6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5dbba6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5db411f9d0 con 0x7f5db409edf0 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5db9809640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5db9809640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54402/0 (socket says 172.21.15.133:54402) 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5db9809640 1 -- 172.21.15.133:0/1223204047 learned_addr learned my addr 172.21.15.133:0/1223204047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-02T14:28:25.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5db9809640 1 -- 172.21.15.133:0/1223204047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4121fa0 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.206+0000 7f5db9809640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 crc :-1 s=READY pgs=12351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.207+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5db000e040 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.207+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5db0012a90 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.207+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db4069090 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.207+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5db0012d30 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.207+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db4131e60 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.208+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 45834+0+0 (crc 0 0 0) 0x7f5db00153f0 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.208+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4068e30 con 0x7f5db409edf0 2024-04-02T14:28:25.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.208+0000 7f5daa7fc640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 0x7f5d9803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:28:25.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.208+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(287..287 src has 237..287) ==== 3424+0+0 (crc 0 0 0) 0x7f5db0049c90 con 0x7f5db409edf0 2024-04-02T14:28:25.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.208+0000 7f5db9008640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 0x7f5d9803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:28:25.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.209+0000 7f5db9008640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 0x7f5d9803ae90 crc :-1 s=READY pgs=3812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:28:25.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.213+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (crc 0 0 0) 0x7f5db000fe60 con 0x7f5db409edf0 2024-04-02T14:28:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.394+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5db4069090 con 0x7f5db409edf0 2024-04-02T14:28:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.397+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (crc 0 0 0) 0x7f5db4069090 con 0x7f5db409edf0 2024-04-02T14:28:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.397+0000 7f5daa7fc640 1 -- 172.21.15.133:0/1223204047 <== mon.0 v2:172.21.15.133:3300/0 8 ==== config(1 keys) ==== 35+0+0 (crc 0 0 0) 0x7f5db000fe60 con 0x7f5db409edf0 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 msgr2=0x7f5d9803ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 0x7f5d9803ae90 crc :-1 s=READY pgs=3812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 msgr2=0x7f5db411dea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 crc :-1 s=READY pgs=12351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 shutdown_connections 2024-04-02T14:28:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.138:6800/1817191150,v1:172.21.15.138:6801/1817191150] conn(0x7f5d98038980 0x7f5d9803ae90 unknown :-1 s=CLOSED pgs=3812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 --2- 172.21.15.133:0/1223204047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5db409edf0 0x7f5db411dea0 unknown :-1 s=CLOSED pgs=12351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:28:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 >> 172.21.15.133:0/1223204047 conn(0x7f5db40a3830 msgr2=0x7f5db41bc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:28:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 shutdown_connections 2024-04-02T14:28:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-02T14:28:25.399+0000 7f5dbba6d640 1 -- 172.21.15.133:0/1223204047 wait complete. 2024-04-02T14:28:25.414 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-02T14:28:25.414 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-02T14:28:25.415 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-02T14:28:25.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.494 INFO:tasks.ceph.osd.0:Stopped 2024-04-02T14:28:25.494 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-02T14:28:25.494 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.570 INFO:tasks.ceph.osd.1:Stopped 2024-04-02T14:28:25.570 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-02T14:28:25.570 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.642 INFO:tasks.ceph.osd.2:Stopped 2024-04-02T14:28:25.642 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-02T14:28:25.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.711 INFO:tasks.ceph.osd.3:Stopped 2024-04-02T14:28:25.712 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-02T14:28:25.712 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-02T14:28:25.712 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.740 INFO:tasks.ceph.mgr.x:Stopped 2024-04-02T14:28:25.740 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-02T14:28:25.741 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-02T14:28:25.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T14:28:25.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-02T14:28:25.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-04-02T14:28:25.774 INFO:tasks.ceph.mon.a:Stopped 2024-04-02T14:28:25.774 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-02T14:28:25.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-02T14:28:25.852 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T14:28:25.853 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-02T14:28:26.001 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T14:28:26.001 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-02T14:28:26.091 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T14:28:26.091 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-02T14:28:26.158 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T14:28:26.159 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-02T14:28:26.227 INFO:tasks.ceph:Archiving mon data... 2024-04-02T14:28:26.229 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/data/mon.a.tgz 2024-04-02T14:28:26.229 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-04-02T14:28:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.MMbQU8tH0b 2024-04-02T14:28:26.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MMbQU8tH0b 2024-04-02T14:28:26.377 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.MMbQU8tH0b 2024-04-02T14:28:26.465 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.MMbQU8tH0b is 338KB 2024-04-02T14:28:26.561 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.MMbQU8tH0b 2024-04-02T14:28:26.576 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-02T14:28:26.576 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-02T14:28:26.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-02T14:28:26.696 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-02T14:28:26.712 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-02T14:28:26.712 INFO:tasks.ceph:Archiving crash dumps... 2024-04-02T14:28:26.714 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi133/crash 2024-04-02T14:28:26.717 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-02T14:28:26.764 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi138/crash 2024-04-02T14:28:26.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-02T14:28:26.805 INFO:tasks.ceph:Compressing logs... 2024-04-02T14:28:26.806 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:28:26.809 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:28:26.852 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96589.log 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32467.log 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32030.log 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32467.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.96589.log.gz 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32466.log 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32467.log.gz 2024-04-02T14:28:26.853 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32576.log 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32030.log.gz 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96211.log 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32576.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32466.log.gz 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.32576.log.gz 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31122.log 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.96211.log.gz 2024-04-02T14:28:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31122.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96547.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.31122.log.gz 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32879.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32660.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.96547.log.gz 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32198.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31151.log 2024-04-02T14:28:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32660.log: /var/log/ceph/ceph-client.admin.32198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96715.log 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32660.log.gz 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32198.log.gz 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31151.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.31151.log.gz 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32156.log 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32114.log 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.1.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.96715.log.gz 2024-04-02T14:28:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34124.log 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.32156.log.gz 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95939.log 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34124.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32114.log.gz 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96421.log 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.95939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34177.log 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34124.log.gz 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96169.log 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.96421.log.gz 2024-04-02T14:28:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33348.log 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30966.log 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34177.log.gz 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr: 85.5%/var/log/ceph/ceph-client.admin.33348.log: -- replaced with /var/log/ceph/ceph-client.admin.96169.log.gz 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32072.log 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5/var/log/ceph/ceph-client.admin.30966.log: --verbose -- 92.4% /var/log/ceph/ceph.tmp-client.admin.28579.log 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32072.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30966.log.gz 2024-04-02T14:28:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32468.log 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.32072.log.gz 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.tmp-client.admin.28579.log: gzip -5 --verbose 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28579.log.gz 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.96337.log 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30887.log 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32468.log.gz 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32618.log 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.96337.log.gz/var/log/ceph/ceph-client.admin.30887.log: 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96253.log 2024-04-02T14:28:26.859 INFO:teuthology.orchestra.run.smithi133.stderr: 86.2%/var/log/ceph/ceph-client.admin.32618.log: -- replaced with /var/log/ceph/ceph-client.admin.30887.log.gz 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96631.log 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96253.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.32618.log.gz 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34292.log 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.96253.log.gz 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96631.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.96631.log.gz 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32300.log 2024-04-02T14:28:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34158.log 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.32300.log.gz 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96673.log 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34158.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.34158.log.gz 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96505.log 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96673.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.96673.log.gz 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96755.log 2024-04-02T14:28:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96505.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.96505.log.gz 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32465.log 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96755.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.96755.log.gz 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34216.log 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32465.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32465.log.gz 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32298.log 2024-04-02T14:28:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34216.log: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.33348.log.gz 2024-04-02T14:28:26.868 INFO:teuthology.orchestra.run.smithi133.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.95939.log.gz 2024-04-02T14:28:26.868 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32299.log 2024-04-02T14:28:26.868 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96463.log 2024-04-02T14:28:26.868 INFO:teuthology.orchestra.run.smithi133.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.32298.log.gz 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32299.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32299.log.gz 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96379.log 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34254.log 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.96463.log.gz 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34273.log 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96295.log 2024-04-02T14:28:26.869 INFO:teuthology.orchestra.run.smithi133.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.96379.log.gz 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31974.log 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34254.log.gz 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.96295.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34273.log.gz 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.96295.log.gz 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31974.log: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.34216.log.gz 2024-04-02T14:28:26.870 INFO:teuthology.orchestra.run.smithi133.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.31974.log.gz 2024-04-02T14:28:26.880 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30581.log 2024-04-02T14:28:26.880 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30701.log 2024-04-02T14:28:26.880 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30661.log 2024-04-02T14:28:26.880 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.30581.log: /var/log/ceph/ceph-client.admin.30701.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.30581.log.gz 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.30701.log.gz 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.30661.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.30661.log.gz 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30621.log 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-02T14:28:26.881 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.30621.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30621.log.gz 2024-04-02T14:28:27.280 INFO:teuthology.orchestra.run.smithi133.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34292.log.gz 2024-04-02T14:28:27.771 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.2.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-02T14:28:29.969 INFO:teuthology.orchestra.run.smithi138.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-02T14:28:30.103 INFO:teuthology.orchestra.run.smithi133.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-02T14:28:30.245 INFO:teuthology.orchestra.run.smithi133.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-02T14:28:30.384 INFO:teuthology.orchestra.run.smithi138.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-02T14:28:30.385 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-02T14:28:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m3.520s 2024-04-02T14:28:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m7.142s 2024-04-02T14:28:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.358s 2024-04-02T14:28:30.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-02T14:28:30.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-04-02T14:28:31.086 INFO:teuthology.orchestra.run.smithi133.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-02T14:28:31.088 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-02T14:28:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m4.267s 2024-04-02T14:28:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m10.967s 2024-04-02T14:28:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m0.498s 2024-04-02T14:28:31.088 INFO:tasks.ceph:Archiving logs... 2024-04-02T14:28:31.089 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi133/log 2024-04-02T14:28:31.090 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:28:32.892 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi138/log 2024-04-02T14:28:32.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:28:34.011 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T14:28:34.020 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T14:28:34.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:28:34.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:28:34.112 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T14:28:34.122 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T14:28:34.122 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:28:34.125 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:28:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:28:34.139 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:28:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:28:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-02T14:28:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 78 -530us[ -530us] +/- 86ms 2024-04-02T14:28:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 8 377 468 -1557us[-1320us] +/- 78ms 2024-04-02T14:28:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 77 +2112us[+2112us] +/- 76ms 2024-04-02T14:28:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:34.145 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:28:34.145 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-02T14:28:34.145 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 -458us[ -458us] +/- 66ms 2024-04-02T14:28:34.145 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -1439us[-1439us] +/- 55ms 2024-04-02T14:28:34.146 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 1 +2221us[+2221us] +/- 77ms 2024-04-02T14:28:34.146 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:28:34.146 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T14:28:34.155 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T14:28:34.155 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T14:28:34.164 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:28:34.253 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:28:34.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:28:34.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:28:34.350 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:28:34.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:28:34.379 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T14:28:34.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T14:28:34.399 INFO:teuthology.task.internal:Duration was 11397.477108 seconds 2024-04-02T14:28:34.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T14:28:34.408 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T14:28:34.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:28:34.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:28:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:28:34.462 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:28:34.663 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T14:28:34.663 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-04-02T14:28:34.663 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:28:34.722 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-02T14:28:34.723 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:28:34.745 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T14:28:34.746 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:28:34.765 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:28:34.814 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T14:28:34.814 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:28:34.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:28:34.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T14:28:34.963 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T14:28:34.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:28:34.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:28:35.017 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T14:28:35.028 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:28:35.033 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:28:35.059 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-04-02T14:28:35.084 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-02T14:28:35.136 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:28:35.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:28:35.150 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:28:35.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:28:35.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T14:28:35.200 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T14:28:35.201 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi133 2024-04-02T14:28:35.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:28:35.239 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494/remote/smithi138 2024-04-02T14:28:35.240 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:28:35.274 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T14:28:35.275 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:28:35.277 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:28:35.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T14:28:35.335 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T14:28:35.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T14:28:35.345 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T14:28:35.345 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:28:35.348 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:28:35.359 INFO:teuthology.orchestra.run.smithi133.stdout: 265317 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 2 14:28 /home/ubuntu/cephtest 2024-04-02T14:28:35.360 INFO:teuthology.orchestra.run.smithi133.stdout: 403401 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 2 11:26 /home/ubuntu/cephtest/mnt.0 2024-04-02T14:28:35.360 INFO:teuthology.orchestra.run.smithi133.stdout: 403402 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 2 11:27 /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T14:28:35.360 INFO:teuthology.orchestra.run.smithi133.stdout: 417741 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 11:27 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T14:28:35.361 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-02T14:28:35.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:28:35.368 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi133 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-02T14:28:35.377 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T14:28:35.387 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T14:28:35.398 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi133 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-02T14:28:35.398 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T14:28:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:28 /home/ubuntu/cephtest 2024-04-02T14:28:35.426 DEBUG:teuthology.nuke:shortname: smithi133 2024-04-02T14:28:35.426 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:28:35.447 DEBUG:teuthology.nuke:shortname: smithi138 2024-04-02T14:28:35.448 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:28:35.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 11:10:15.409715', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJK+nFThnXLgUX2cmMcAQA4Ue5WcKpH70qrYfnnQQBrUA/tKnCFvCD3dWiMXtv168DqA/XIP9BUTn5gZh8XTUc='} 2024-04-02T14:28:35.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-04-02_11:03:04-crimson-rados-wip-matanb-crimson-testing-apr-2-distro-crimson-smithi/7635494', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 11:10:15.408007', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBILXb1sCtBCsjkTFXqvIZz4kqBeJ8pjSF1jv1Hq7ql4vvYJOoCxP9hUF/7C9k5YUBVahFiqqeq05+Sz6GbqJOyU='} 2024-04-02T14:28:35.502 INFO:teuthology.orchestra.console.smithi133:Power off 2024-04-02T14:28:35.503 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:28:35.537 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-02T14:28:35.538 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:28:35.548 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-04-02T14:28:35.548 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:35.563 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-02T14:28:35.563 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:35.575 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-02T14:28:35.588 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-02T14:28:39.576 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:39.588 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:39.688 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-02T14:28:39.700 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-02T14:28:43.690 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:43.701 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:43.803 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-02T14:28:43.812 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-02T14:28:47.806 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:47.817 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:28:47.919 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-04-02T14:28:47.934 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-02T14:28:48.021 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-04-02T14:28:48.036 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-02T14:28:48.143 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 11397.477107524872 failure_reason: 'Command failed (workunit test rados/test.sh) on smithi133 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=107e1624f2a9b72a56ee529969558944d24a2a75 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson''' flavor: crimson owner: scheduled_matan@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1e9da25bf98944319d992954582d67cf status: fail success: false 2024-04-02T14:28:48.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:28:48.241 INFO:teuthology.run:FAIL