2024-02-20T01:57:01.772 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-20T01:57:01.773 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_sjust@teuthology --archive /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238 --name sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} -- /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/orig.config.yaml 2024-02-20T01:57:01.802 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-20T01:57:01.879 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238 branch: wip-sjust-crimson-testing-2024-02-19 description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7568238' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-sjust-crimson-testing-2024-02-19 ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c workunit: branch: wip-sjust-crimson-testing-2024-02-19 env: CRIMSON_COMPAT: '1' sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c owner: scheduled_sjust@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 4769 sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-sjust-crimson-testing-2024-02-19 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c targets: smithi125.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Qp7DETLHvlRbXOx2wpU4DdRiTNXdGoP1rxNFm4cBMOomqjvMx+/OclkALynycfzFVlOMbvPgxUrIL4pI3mi8= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Q8rcGTT41i9niKJxNu8yqnottToCTCJw8aBW7rEmy1tCZ0L0RzXTjJ4/8mOQVVvrgN1ksvraVhD+S5Pp337M= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-20_01:50:29 tube: smithi user: sjust verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-20T01:57:01.880 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa; will attempt to use it 2024-02-20T01:57:01.880 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa/tasks 2024-02-20T01:57:01.880 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-20T01:57:01.882 INFO:teuthology.task.internal:Checking packages... 2024-02-20T01:57:01.905 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '663dfa8925d03992983e2ae809f7a5fc9835cd6c' 2024-02-20T01:57:01.905 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T01:57:01.905 INFO:teuthology.packaging:ref: None 2024-02-20T01:57:01.905 INFO:teuthology.packaging:tag: None 2024-02-20T01:57:01.905 INFO:teuthology.packaging:branch: wip-sjust-crimson-testing-2024-02-19 2024-02-20T01:57:01.905 INFO:teuthology.packaging:sha1: 663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T01:57:01.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-sjust-crimson-testing-2024-02-19 2024-02-20T01:57:02.124 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1579.g663dfa89 2024-02-20T01:57:02.126 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-20T01:57:02.133 INFO:teuthology.task.internal:no buildpackages task found 2024-02-20T01:57:02.133 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-20T01:57:02.175 INFO:teuthology.task.internal:Saving configuration 2024-02-20T01:57:02.187 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-20T01:57:02.209 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T01:57:02.234 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 01:51:44.834395', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Qp7DETLHvlRbXOx2wpU4DdRiTNXdGoP1rxNFm4cBMOomqjvMx+/OclkALynycfzFVlOMbvPgxUrIL4pI3mi8='} 2024-02-20T01:57:02.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 01:51:44.835825', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Q8rcGTT41i9niKJxNu8yqnottToCTCJw8aBW7rEmy1tCZ0L0RzXTjJ4/8mOQVVvrgN1ksvraVhD+S5Pp337M='} 2024-02-20T01:57:02.258 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-20T01:57:02.264 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-02-20T01:57:02.264 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-02-20T01:57:02.264 INFO:teuthology.run_tasks:Running task console_log... 2024-02-20T01:57:02.356 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff204d03e50>, signals=[15]) 2024-02-20T01:57:02.357 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-20T01:57:02.363 INFO:teuthology.task.internal:Opening connections... 2024-02-20T01:57:02.364 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T01:57:02.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:57:02.454 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T01:57:02.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:57:02.543 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-20T01:57:02.550 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2024-02-20T01:57:02.568 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2024-02-20T01:57:02.568 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2024-02-20T01:57:02.624 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2024-02-20T01:57:02.624 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2024-02-20T01:57:02.624 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-20T01:57:02.625 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-20T01:57:02.626 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2024-02-20T01:57:02.651 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-02-20T01:57:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-02-20T01:57:02.668 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-02-20T01:57:02.725 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-20T01:57:02.726 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-20T01:57:02.726 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-02-20T01:57:02.751 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-20T01:57:02.760 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-20T01:57:02.793 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-20T01:57:02.793 DEBUG:teuthology.orchestra.run.smithi125:> test '!' -e /home/ubuntu/cephtest 2024-02-20T01:57:02.795 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-02-20T01:57:02.811 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-20T01:57:02.827 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-20T01:57:02.827 DEBUG:teuthology.orchestra.run.smithi125:> test -z $(ls -A /var/lib/ceph) 2024-02-20T01:57:02.852 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-02-20T01:57:02.929 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-20T01:57:02.984 INFO:teuthology.run_tasks:Running task kernel... 2024-02-20T01:57:02.999 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-20T01:57:02.999 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-20T01:57:02.999 DEBUG:teuthology.orchestra.run.smithi125:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-20T01:57:03.000 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-20T01:57:03.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T01:57:03.016 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-02-20T01:57:03.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T01:57:03.017 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2024-02-20T01:57:03.072 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-361.el9.x86_64 2024-02-20T01:57:03.072 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.14.0-361.el9.x86_64 2024-02-20T01:57:03.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum install -y kernel 2024-02-20T01:57:03.073 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-361.el9.x86_64 2024-02-20T01:57:03.073 INFO:teuthology.task.kernel:Running kernel on smithi125: 5.14.0-361.el9.x86_64 2024-02-20T01:57:03.073 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum install -y kernel 2024-02-20T01:57:05.332 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-02-20T01:57:05.769 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 6.8 MB/s | 6.7 MB 00:00 2024-02-20T01:57:07.007 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-02-20T01:57:07.436 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-02-20T01:57:11.514 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-02-20T01:57:17.215 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 19 MB 00:04 2024-02-20T01:57:21.708 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 587 kB/s | 8.0 MB 00:13 2024-02-20T01:57:23.870 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-02-20T01:57:28.011 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-02-20T01:57:29.355 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 720 kB/s | 6.1 MB 00:08 2024-02-20T01:57:29.603 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-20T01:57:30.362 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 32 MB/s | 20 MB 00:00 2024-02-20T01:57:30.881 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-02-20T01:57:32.624 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-20T01:57:35.187 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-02-20T01:57:37.411 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-20T01:57:37.411 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-20T01:57:37.457 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-02-20T01:57:37.541 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-20T01:57:37.551 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:57:37.552 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T01:57:37.552 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T01:57:37.552 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-02-20T01:57:37.552 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:57:37.552 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 100 M 2024-02-20T01:57:37.553 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 123 M 2024-02-20T01:57:37.553 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T01:57:38.172 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 9.8 MB/s | 6.1 MB 00:00 2024-02-20T01:57:38.672 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-20T01:57:39.397 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-02-20T01:57:39.730 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-20T01:57:39.730 INFO:teuthology.orchestra.run.smithi125.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-20T01:57:39.822 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T01:57:39.833 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T01:57:39.833 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T01:57:39.833 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T01:57:39.833 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout:Install 4 Packages 2024-02-20T01:57:39.834 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:57:39.835 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 100 M 2024-02-20T01:57:39.835 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 123 M 2024-02-20T01:57:39.835 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T01:57:39.889 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 20 MB/s | 33 MB 00:01 2024-02-20T01:57:39.892 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T01:57:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:Total 43 MB/s | 100 MB 00:02 2024-02-20T01:57:39.893 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T01:57:40.184 INFO:teuthology.orchestra.run.smithi125.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 17 MB/s | 6.1 MB 00:00 2024-02-20T01:57:40.263 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T01:57:40.263 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T01:57:40.725 INFO:teuthology.orchestra.run.smithi125.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 23 MB/s | 21 MB 00:00 2024-02-20T01:57:41.295 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T01:57:41.296 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T01:57:41.525 INFO:teuthology.orchestra.run.smithi125.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 25 MB/s | 33 MB 00:01 2024-02-20T01:57:41.917 INFO:teuthology.orchestra.run.smithi125.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 19 MB/s | 39 MB 00:02 2024-02-20T01:57:41.920 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T01:57:41.921 INFO:teuthology.orchestra.run.smithi125.stdout:Total 48 MB/s | 100 MB 00:02 2024-02-20T01:57:41.921 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T01:57:42.297 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T01:57:42.298 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T01:57:43.314 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T01:57:43.315 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T01:57:43.738 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T01:57:44.515 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-20T01:57:44.649 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:57:45.727 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T01:57:46.556 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-20T01:57:46.692 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:57:47.428 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:57:47.612 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:57:49.477 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:57:49.651 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:57:52.330 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:57:52.541 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:57:54.353 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:57:54.639 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:57:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:57:59.357 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:26.655 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:26.655 INFO:teuthology.orchestra.run.smithi204.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-20T01:58:26.655 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:58:26.724 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:27.160 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:27.160 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-20T01:58:27.160 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:58:27.160 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:58:27.563 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T01:58:28.082 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T01:58:28.212 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:28.212 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-20T01:58:28.212 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-20T01:58:28.212 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-20T01:58:28.212 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-20T01:58:28.213 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-02-20T01:58:28.213 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-02-20T01:58:28.213 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T01:58:28.320 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:28.320 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-20T01:58:28.321 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-02-20T01:58:28.354 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-02-20T01:58:28.355 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T01:58:28.356 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-02-20T01:58:28.357 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-20T01:58:28.357 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 5.14.0-421.el9.x86_64 2024-02-20T01:58:28.357 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-20T01:58:28.497 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:28.498 INFO:teuthology.orchestra.run.smithi125.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-20T01:58:28.498 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:58:28.568 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:28.993 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-02-20T01:58:29.020 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:29.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-20T01:58:29.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-20T01:58:29.021 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-20T01:58:29.456 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-20T01:58:29.456 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:58:29.456 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T01:58:29.456 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:29.457 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-20T01:58:29.457 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-20T01:58:29.457 INFO:teuthology.orchestra.run.smithi125.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-20T01:58:29.457 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:58:29.457 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T01:58:29.955 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T01:58:30.083 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:30.087 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-20T01:58:30.087 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-20T01:58:30.087 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-20T01:58:30.087 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-20T01:58:30.087 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-20T01:58:30.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi125.front.sepia.ceph.com, path=None, version=distro) 2024-02-20T01:58:30.088 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T01:58:30.196 INFO:teuthology.orchestra.run.smithi125.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-20T01:58:30.196 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-20T01:58:30.196 DEBUG:teuthology.orchestra.run.smithi125:> sudo rpm -qi grub2-tools 2024-02-20T01:58:30.229 INFO:teuthology.orchestra.run.smithi125.stdout:Name : grub2-tools 2024-02-20T01:58:30.229 INFO:teuthology.orchestra.run.smithi125.stdout:Epoch : 1 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Version : 2.06 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Release : 68.el9 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Architecture: x86_64 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Group : Unspecified 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Size : 8264254 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:License : GPLv3+ 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Packager : builder@centos.org 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Vendor : CentOS 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-20T01:58:30.230 INFO:teuthology.orchestra.run.smithi125.stdout:Summary : Support tools for GRUB. 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:Description : 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:hardware devices. 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T01:58:30.231 INFO:teuthology.orchestra.run.smithi125.stdout:This subpackage provides tools for support of all platforms. 2024-02-20T01:58:30.232 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-20T01:58:30.232 INFO:teuthology.task.kernel:Updating grub on smithi125 to boot 5.14.0-421.el9.x86_64 2024-02-20T01:58:30.232 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-20T01:58:30.319 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-20T01:58:30.360 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-02-20T01:58:30.362 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-02-20T01:58:30.377 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.1OMwCPodbq 2024-02-20T01:58:30.377 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1OMwCPodbq 2024-02-20T01:58:30.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.1OMwCPodbq 2024-02-20T01:58:30.589 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.1OMwCPodbq is 6KB 2024-02-20T01:58:30.638 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.1OMwCPodbq 2024-02-20T01:58:30.652 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-02-20T01:58:30.727 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-20T01:58:30.727 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-20T01:58:30.727 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-20T01:58:30.728 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-20T01:58:30.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-20T01:58:30.867 INFO:teuthology.orchestra.run.smithi125.stderr:Generating grub configuration file ... 2024-02-20T01:58:30.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-02-20T01:58:32.306 INFO:teuthology.orchestra.run.smithi125.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-20T01:58:32.340 INFO:teuthology.orchestra.run.smithi125.stderr:done 2024-02-20T01:58:32.342 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2024-02-20T01:58:32.358 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.zpJdKLdlQL 2024-02-20T01:58:32.358 DEBUG:teuthology.orchestra.run.smithi125:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zpJdKLdlQL 2024-02-20T01:58:32.427 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.zpJdKLdlQL 2024-02-20T01:58:32.568 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.zpJdKLdlQL is 6KB 2024-02-20T01:58:32.618 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.zpJdKLdlQL 2024-02-20T01:58:32.632 DEBUG:teuthology.orchestra.run.smithi125:> sudo /bin/ls /boot/loader/entries || true 2024-02-20T01:58:32.701 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-20T01:58:32.702 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-20T01:58:32.702 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-20T01:58:32.702 INFO:teuthology.orchestra.run.smithi125.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-20T01:58:32.703 DEBUG:teuthology.orchestra.run.smithi125:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-20T01:58:32.858 DEBUG:teuthology.orchestra.run.smithi125:> sudo shutdown -r now 2024-02-20T01:59:00.926 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-02-20T01:59:00.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T01:59:00.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:59:02.862 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-20T01:59:02.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T01:59:02.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:59:19.356 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-02-20T01:59:21.404 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-02-20T01:59:28.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T01:59:28.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:59:30.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T01:59:30.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:59:31.420 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-02-20T01:59:33.468 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-02-20T01:59:43.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T01:59:43.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T01:59:45.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T01:59:45.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T02:00:03.900 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-02-20T02:00:18.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T02:00:18.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T02:00:19.314 DEBUG:teuthology.orchestra.run.smithi125:> true 2024-02-20T02:00:19.717 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T02:00:19.717 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-421.el9.x86_64"... 2024-02-20T02:00:19.718 DEBUG:teuthology.orchestra.run.smithi125:> uname -r 2024-02-20T02:00:19.732 INFO:teuthology.orchestra.run.smithi125.stdout:5.14.0-421.el9.x86_64 2024-02-20T02:00:19.732 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-20T02:00:19.732 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-20T02:00:19.732 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-20T02:00:20.732 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-20T02:00:20.733 DEBUG:teuthology.orchestra.run.smithi125:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-20T02:00:20.790 INFO:teuthology.orchestra.run.smithi125.stdout:ttyS1 2024-02-20T02:00:20.822 DEBUG:teuthology.parallel:result is None 2024-02-20T02:00:43.433 DEBUG:teuthology.orchestra.remote:timed out 2024-02-20T02:00:58.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T02:00:58.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T02:00:58.852 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-02-20T02:00:59.271 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T02:00:59.272 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-421.el9.x86_64"... 2024-02-20T02:00:59.272 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-02-20T02:00:59.287 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-421.el9.x86_64 2024-02-20T02:00:59.288 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-20T02:00:59.288 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-20T02:00:59.288 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-20T02:01:00.288 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-02-20T02:01:00.289 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-20T02:01:00.367 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-02-20T02:01:00.402 DEBUG:teuthology.parallel:result is None 2024-02-20T02:01:00.403 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-20T02:01:00.412 INFO:teuthology.task.internal:Creating test directory... 2024-02-20T02:01:00.412 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-20T02:01:00.415 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-20T02:01:00.432 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-20T02:01:00.463 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-20T02:01:00.512 INFO:teuthology.task.internal:Creating archive directory... 2024-02-20T02:01:00.512 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-20T02:01:00.514 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-20T02:01:00.549 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-20T02:01:00.558 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-20T02:01:00.558 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-20T02:01:00.587 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-20T02:01:00.623 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T02:01:00.626 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T02:01:00.637 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T02:01:00.640 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T02:01:00.642 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-20T02:01:00.649 INFO:teuthology.task.internal:Configuring sudo... 2024-02-20T02:01:00.649 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-20T02:01:00.684 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-20T02:01:00.713 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-20T02:01:00.723 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-20T02:01:00.724 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-20T02:01:00.754 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-20T02:01:00.769 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T02:01:00.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T02:01:00.900 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T02:01:00.983 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T02:01:01.025 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:01:01.025 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-20T02:01:01.091 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T02:01:01.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T02:01:01.183 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T02:01:01.267 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T02:01:01.312 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:01:01.312 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-20T02:01:01.379 DEBUG:teuthology.orchestra.run.smithi125:> sudo service rsyslog restart 2024-02-20T02:01:01.382 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-02-20T02:01:01.436 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T02:01:01.466 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T02:01:01.894 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-20T02:01:01.903 INFO:teuthology.task.internal:Starting timer... 2024-02-20T02:01:01.903 INFO:teuthology.run_tasks:Running task pcp... 2024-02-20T02:01:01.955 INFO:teuthology.run_tasks:Running task selinux... 2024-02-20T02:01:01.987 DEBUG:teuthology.orchestra.run.smithi125:> sudo service auditd rotate 2024-02-20T02:01:02.056 INFO:teuthology.orchestra.run.smithi125.stdout:Rotating logs: 2024-02-20T02:01:02.058 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-02-20T02:01:02.113 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-02-20T02:01:02.115 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-20T02:01:02.115 DEBUG:teuthology.orchestra.run.smithi125:> /usr/sbin/getenforce 2024-02-20T02:01:02.138 INFO:teuthology.orchestra.run.smithi125.stdout:Permissive 2024-02-20T02:01:02.138 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-02-20T02:01:02.159 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-02-20T02:01:02.160 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi125.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-02-20T02:01:02.160 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T02:01:02.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:01:02.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T02:01:02.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:01:02.234 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-20T02:01:02.234 DEBUG:teuthology.orchestra.run.smithi125:> sudo /usr/sbin/setenforce permissive 2024-02-20T02:01:02.282 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-02-20T02:01:02.311 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-20T02:01:02.321 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-20T02:01:02.327 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-20T02:01:02.481 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-20T02:01:02.495 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-20T02:01:02.497 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi125.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-20T02:05:13.848 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi125.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-02-20T02:05:13.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T02:05:13.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T02:05:13.940 DEBUG:teuthology.orchestra.run.smithi125:> true 2024-02-20T02:05:14.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-20T02:05:14.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T02:05:14.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T02:05:14.110 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-02-20T02:05:14.186 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-02-20T02:05:14.187 INFO:teuthology.run_tasks:Running task clock... 2024-02-20T02:05:14.198 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-20T02:05:14.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-20T02:05:14.199 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T02:05:14.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-20T02:05:14.202 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T02:05:14.230 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-20T02:05:14.248 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-20T02:05:14.258 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-20T02:05:14.275 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-20T02:05:14.277 INFO:teuthology.orchestra.run.smithi125.stderr:sudo: ntpd: command not found 2024-02-20T02:05:14.291 INFO:teuthology.orchestra.run.smithi125.stdout:506 Cannot talk to daemon 2024-02-20T02:05:14.304 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-02-20T02:05:14.307 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-20T02:05:14.317 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-02-20T02:05:14.323 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-20T02:05:14.334 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-20T02:05:14.350 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-20T02:05:14.375 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2024-02-20T02:05:14.378 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T02:05:14.378 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2024-02-20T02:05:14.378 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.378 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.378 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.379 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.412 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.416 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:05:14.417 INFO:teuthology.run_tasks:Running task install... 2024-02-20T02:05:14.429 DEBUG:teuthology.task.install:project ceph 2024-02-20T02:05:14.429 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c'}} 2024-02-20T02:05:14.429 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c'} 2024-02-20T02:05:14.429 INFO:teuthology.task.install:Using flavor: crimson 2024-02-20T02:05:14.439 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-20T02:05:14.439 INFO:teuthology.task.install:extra packages: [] 2024-02-20T02:05:14.439 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c', 'tag': None, 'wait_for_package': False} 2024-02-20T02:05:14.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:05:14.442 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c', 'tag': None, 'wait_for_package': False} 2024-02-20T02:05:14.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:05:14.603 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/ 2024-02-20T02:05:14.604 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1579.g663dfa89 2024-02-20T02:05:14.716 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/ 2024-02-20T02:05:14.716 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1579.g663dfa89 2024-02-20T02:05:14.814 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-20T02:05:14.814 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:05:14.814 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-20T02:05:14.842 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-20T02:05:14.842 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/663dfa8925d03992983e2ae809f7a5fc9835cd6c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-20T02:05:14.854 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-crimson-testing-2024-02-19/663dfa8925d03992983e2ae809f7a5fc9835cd6c/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-20T02:05:14.854 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:05:14.855 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-20T02:05:14.883 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-20T02:05:14.883 DEBUG:teuthology.orchestra.run.smithi125:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/663dfa8925d03992983e2ae809f7a5fc9835cd6c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-20T02:05:14.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-20T02:05:14.951 DEBUG:teuthology.orchestra.run.smithi125:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-20T02:05:14.990 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-20T02:05:15.031 DEBUG:teuthology.orchestra.run.smithi125:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-20T02:05:15.057 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-02-20T02:05:15.058 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-02-20T02:05:15.097 INFO:teuthology.orchestra.run.smithi125.stdout:check_obsoletes = 1 2024-02-20T02:05:15.099 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum clean all 2024-02-20T02:05:15.300 INFO:teuthology.orchestra.run.smithi204.stdout:74 files removed 2024-02-20T02:05:15.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw 2024-02-20T02:05:15.352 INFO:teuthology.orchestra.run.smithi125.stdout:74 files removed 2024-02-20T02:05:15.382 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-radosgw 2024-02-20T02:05:16.014 INFO:teuthology.orchestra.run.smithi125.stdout:ceph packages for x86_64 271 kB/s | 85 kB 00:00 2024-02-20T02:05:16.068 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 204 kB/s | 85 kB 00:00 2024-02-20T02:05:16.452 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-02-20T02:05:16.481 INFO:teuthology.orchestra.run.smithi125.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-02-20T02:05:16.848 INFO:teuthology.orchestra.run.smithi125.stdout:ceph source packages 6.1 kB/s | 2.0 kB 00:00 2024-02-20T02:05:16.869 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 5.2 kB/s | 2.0 kB 00:00 2024-02-20T02:05:17.601 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 9.4 MB/s | 6.7 MB 00:00 2024-02-20T02:05:17.880 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 6.9 MB/s | 6.7 MB 00:00 2024-02-20T02:05:19.485 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 2.6 MB/s | 2.0 MB 00:00 2024-02-20T02:05:19.797 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 2.6 MB/s | 2.0 MB 00:00 2024-02-20T02:05:21.038 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-02-20T02:05:21.575 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-02-20T02:05:28.172 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 19 MB 00:06 2024-02-20T02:05:33.198 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-02-20T02:05:34.890 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 15 kB 00:00 2024-02-20T02:05:37.525 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 20 MB 00:02 2024-02-20T02:05:41.240 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-02-20T02:05:42.533 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-02-20T02:05:45.174 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-02-20T02:05:45.203 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 9.1 M 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.2 M 2024-02-20T02:05:45.210 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.1 M 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 5.4 M 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 18 M 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 25 k 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 752 k 2024-02-20T02:05:45.211 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 497 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 4.8 M 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 45 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 131 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 162 k 2024-02-20T02:05:45.212 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 320 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 298 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 99 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:05:45.213 INFO:teuthology.orchestra.run.smithi204.stdout:Install 30 Packages 2024-02-20T02:05:45.214 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 2 Packages 2024-02-20T02:05:45.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:05:45.214 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 55 M 2024-02-20T02:05:45.214 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:05:46.776 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-02-20T02:05:47.659 INFO:teuthology.orchestra.run.smithi204.stdout:(1/32): ceph-base-19.0.0-1579.g663dfa89.el9.x86 2.4 MB/s | 5.4 MB 00:02 2024-02-20T02:05:47.768 INFO:teuthology.orchestra.run.smithi204.stdout:(2/32): ceph-selinux-19.0.0-1579.g663dfa89.el9. 233 kB/s | 25 kB 00:00 2024-02-20T02:05:48.060 INFO:teuthology.orchestra.run.smithi204.stdout:(3/32): libcephfs2-19.0.0-1579.g663dfa89.el9.x8 2.5 MB/s | 752 kB 00:00 2024-02-20T02:05:48.268 INFO:teuthology.orchestra.run.smithi204.stdout:(4/32): libradosstriper1-19.0.0-1579.g663dfa89. 2.3 MB/s | 497 kB 00:00 2024-02-20T02:05:48.369 INFO:teuthology.orchestra.run.smithi204.stdout:(5/32): ceph-radosgw-19.0.0-1579.g663dfa89.el9. 3.1 MB/s | 9.1 MB 00:02 2024-02-20T02:05:48.461 INFO:teuthology.orchestra.run.smithi204.stdout:(6/32): python3-ceph-argparse-19.0.0-1579.g663d 494 kB/s | 45 kB 00:00 2024-02-20T02:05:48.611 INFO:teuthology.orchestra.run.smithi204.stdout:(7/32): python3-ceph-common-19.0.0-1579.g663dfa 876 kB/s | 131 kB 00:00 2024-02-20T02:05:48.711 INFO:teuthology.orchestra.run.smithi204.stdout:(8/32): python3-cephfs-19.0.0-1579.g663dfa89.el 1.6 MB/s | 162 kB 00:00 2024-02-20T02:05:48.836 INFO:teuthology.orchestra.run.smithi204.stdout:(9/32): python3-rados-19.0.0-1579.g663dfa89.el9 2.5 MB/s | 320 kB 00:00 2024-02-20T02:05:48.970 INFO:teuthology.orchestra.run.smithi204.stdout:(10/32): python3-rbd-19.0.0-1579.g663dfa89.el9. 2.2 MB/s | 298 kB 00:00 2024-02-20T02:05:49.170 INFO:teuthology.orchestra.run.smithi204.stdout:(11/32): ceph-common-19.0.0-1579.g663dfa89.el9. 4.9 MB/s | 18 MB 00:03 2024-02-20T02:05:49.196 INFO:teuthology.orchestra.run.smithi204.stdout:(12/32): python3-rgw-19.0.0-1579.g663dfa89.el9. 441 kB/s | 99 kB 00:00 2024-02-20T02:05:49.287 INFO:teuthology.orchestra.run.smithi204.stdout:(13/32): librgw2-19.0.0-1579.g663dfa89.el9.x86_ 4.7 MB/s | 4.8 MB 00:01 2024-02-20T02:05:49.354 INFO:teuthology.orchestra.run.smithi204.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 577 kB/s | 106 kB 00:00 2024-02-20T02:05:49.371 INFO:teuthology.orchestra.run.smithi204.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 44 kB 00:00 2024-02-20T02:05:49.413 INFO:teuthology.orchestra.run.smithi204.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 831 kB/s | 159 kB 00:00 2024-02-20T02:05:49.514 INFO:teuthology.orchestra.run.smithi204.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-02-20T02:05:49.539 INFO:teuthology.orchestra.run.smithi204.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 282 kB 00:00 2024-02-20T02:05:49.564 INFO:teuthology.orchestra.run.smithi204.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 274 kB/s | 41 kB 00:00 2024-02-20T02:05:49.589 INFO:teuthology.orchestra.run.smithi204.stdout:(20/32): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 215 kB 00:00 2024-02-20T02:05:49.614 INFO:teuthology.orchestra.run.smithi204.stdout:(21/32): qatlib-service-23.11.0-1.el9.x86_64.rp 451 kB/s | 34 kB 00:00 2024-02-20T02:05:49.640 INFO:teuthology.orchestra.run.smithi204.stdout:(22/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-02-20T02:05:50.048 INFO:teuthology.orchestra.run.smithi204.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 64 kB/s | 26 kB 00:00 2024-02-20T02:05:51.090 INFO:teuthology.orchestra.run.smithi204.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 47 kB/s | 49 kB 00:01 2024-02-20T02:05:51.291 INFO:teuthology.orchestra.run.smithi204.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 184 kB/s | 309 kB 00:01 2024-02-20T02:05:51.358 INFO:teuthology.orchestra.run.smithi204.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-02-20T02:05:51.883 INFO:teuthology.orchestra.run.smithi204.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 365 kB/s | 192 kB 00:00 2024-02-20T02:05:52.059 INFO:teuthology.orchestra.run.smithi204.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 838 kB 00:00 2024-02-20T02:05:52.567 INFO:teuthology.orchestra.run.smithi204.stdout:(29/32): librados2-19.0.0-1579.g663dfa89.el9.x8 6.2 MB/s | 3.2 MB 00:00 2024-02-20T02:05:52.859 INFO:teuthology.orchestra.run.smithi204.stdout:(30/32): librbd1-19.0.0-1579.g663dfa89.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-02-20T02:05:54.258 INFO:teuthology.orchestra.run.smithi204.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 684 kB/s | 1.6 MB 00:02 2024-02-20T02:05:57.760 INFO:teuthology.orchestra.run.smithi204.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 555 kB/s | 4.4 MB 00:08 2024-02-20T02:05:57.766 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:05:57.767 INFO:teuthology.orchestra.run.smithi204.stdout:Total 4.4 MB/s | 55 MB 00:12 2024-02-20T02:05:57.767 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:05:57.804 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:05:57.804 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:05:58.270 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:05:58.270 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:05:59.731 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:06:00.094 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-20T02:06:00.161 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:00.261 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:00.352 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 3/34 2024-02-20T02:06:00.379 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:00.440 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:00.475 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-02-20T02:06:00.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-02-20T02:06:00.523 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-02-20T02:06:00.539 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-02-20T02:06:00.715 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 8/34 2024-02-20T02:06:00.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 8/34 2024-02-20T02:06:00.774 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-02-20T02:06:00.818 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9. 10/34 2024-02-20T02:06:00.846 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 11/34 2024-02-20T02:06:00.913 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 12/34 2024-02-20T02:06:00.936 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 13/34 2024-02-20T02:06:01.008 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 13/34 2024-02-20T02:06:01.043 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-02-20T02:06:01.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-20T02:06:01.214 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-20T02:06:01.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-20T02:06:01.301 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-20T02:06:01.339 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-20T02:06:01.371 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-20T02:06:01.700 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-20T02:06:01.734 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 22/34 2024-02-20T02:06:01.824 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 22/34 2024-02-20T02:06:01.855 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 23/34 2024-02-20T02:06:01.901 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-20T02:06:02.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-20T02:06:02.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 26/34 2024-02-20T02:06:02.458 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 26/34 2024-02-20T02:06:02.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 27/34 2024-02-20T02:06:02.564 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x8 28/34 2024-02-20T02:06:04.684 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:04.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:05.335 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:05.531 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 30/34 2024-02-20T02:06:05.615 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 30/34 2024-02-20T02:06:05.616 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-20T02:06:05.616 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-20T02:06:05.616 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:05.666 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:11.651 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 843 kB/s | 20 MB 00:24 2024-02-20T02:06:15.286 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:15.286 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-02-20T02:06:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-02-20T02:06:15.288 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:16.210 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 32/34 2024-02-20T02:06:16.240 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 32/34 2024-02-20T02:06:16.241 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:06:16.241 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-20T02:06:16.241 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-20T02:06:16.241 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-20T02:06:16.241 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:16.243 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-20T02:06:16.330 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-20T02:06:16.330 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:16.540 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 1/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 3/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 5/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 6/34 2024-02-20T02:06:17.582 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 7/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9. 8/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x8 9/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 10/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 11/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 12/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 13/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-20T02:06:17.583 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-20T02:06:17.584 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-20T02:06:17.584 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-20T02:06:17.584 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-20T02:06:17.584 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-20T02:06:17.585 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 33/34 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.700 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.701 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:18.702 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:06:19.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-test 2024-02-20T02:06:19.159 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 9.1 M 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrading: 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout: librados2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.2 M 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.1 M 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:06:19.212 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 5.4 M 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 18 M 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 25 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 752 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 497 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-20T02:06:19.213 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2 x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 4.8 M 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 45 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 131 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 162 k 2024-02-20T02:06:19.214 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 320 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 298 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 99 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout:Installing weak dependencies: 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-20T02:06:19.215 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:19.216 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:06:19.216 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:19.216 INFO:teuthology.orchestra.run.smithi125.stdout:Install 30 Packages 2024-02-20T02:06:19.216 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrade 2 Packages 2024-02-20T02:06:19.216 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:19.217 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 55 M 2024-02-20T02:06:19.217 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:06:19.599 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:37 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:06:19.770 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:06:19.771 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:06:19.771 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:06:19.771 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:06:19.771 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 122 M 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: libasan x86_64 11.4.1-3.el9 CentOS-AppStream 417 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 158 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: libubsan x86_64 11.4.1-3.el9 CentOS-AppStream 186 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-02-20T02:06:19.772 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:19.773 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:06:19.773 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:06:19.773 INFO:teuthology.orchestra.run.smithi204.stdout:Install 9 Packages 2024-02-20T02:06:19.773 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:06:19.774 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 124 M 2024-02-20T02:06:19.774 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 1.1 G 2024-02-20T02:06:19.774 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:06:20.299 INFO:teuthology.orchestra.run.smithi204.stdout:(1/9): libasan-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 417 kB 00:00 2024-02-20T02:06:20.366 INFO:teuthology.orchestra.run.smithi204.stdout:(2/9): libubsan-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-20T02:06:20.450 INFO:teuthology.orchestra.run.smithi204.stdout:(3/9): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-02-20T02:06:20.483 INFO:teuthology.orchestra.run.smithi204.stdout:(4/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-20T02:06:20.542 INFO:teuthology.orchestra.run.smithi204.stdout:(5/9): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-02-20T02:06:20.592 INFO:teuthology.orchestra.run.smithi204.stdout:(6/9): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-02-20T02:06:20.826 INFO:teuthology.orchestra.run.smithi204.stdout:(7/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 521 kB/s | 122 kB 00:00 2024-02-20T02:06:21.032 INFO:teuthology.orchestra.run.smithi125.stdout:(1/32): ceph-base-19.0.0-1579.g663dfa89.el9.x86 3.5 MB/s | 5.4 MB 00:01 2024-02-20T02:06:21.034 INFO:teuthology.orchestra.run.smithi204.stdout:(8/9): libcephsqlite-19.0.0-1579.g663dfa89.el9. 145 kB/s | 158 kB 00:01 2024-02-20T02:06:21.091 INFO:teuthology.orchestra.run.smithi125.stdout:(2/32): ceph-selinux-19.0.0-1579.g663dfa89.el9. 432 kB/s | 25 kB 00:00 2024-02-20T02:06:21.241 INFO:teuthology.orchestra.run.smithi125.stdout:(3/32): ceph-radosgw-19.0.0-1579.g663dfa89.el9. 5.2 MB/s | 9.1 MB 00:01 2024-02-20T02:06:21.383 INFO:teuthology.orchestra.run.smithi125.stdout:(4/32): libradosstriper1-19.0.0-1579.g663dfa89. 3.4 MB/s | 497 kB 00:00 2024-02-20T02:06:21.425 INFO:teuthology.orchestra.run.smithi125.stdout:(5/32): libcephfs2-19.0.0-1579.g663dfa89.el9.x8 2.2 MB/s | 752 kB 00:00 2024-02-20T02:06:21.500 INFO:teuthology.orchestra.run.smithi125.stdout:(6/32): python3-ceph-argparse-19.0.0-1579.g663d 600 kB/s | 45 kB 00:00 2024-02-20T02:06:21.584 INFO:teuthology.orchestra.run.smithi125.stdout:(7/32): python3-ceph-common-19.0.0-1579.g663dfa 1.5 MB/s | 131 kB 00:00 2024-02-20T02:06:21.684 INFO:teuthology.orchestra.run.smithi125.stdout:(8/32): python3-cephfs-19.0.0-1579.g663dfa89.el 1.6 MB/s | 162 kB 00:00 2024-02-20T02:06:21.859 INFO:teuthology.orchestra.run.smithi125.stdout:(9/32): python3-rados-19.0.0-1579.g663dfa89.el9 1.8 MB/s | 320 kB 00:00 2024-02-20T02:06:22.018 INFO:teuthology.orchestra.run.smithi125.stdout:(10/32): python3-rbd-19.0.0-1579.g663dfa89.el9. 1.8 MB/s | 298 kB 00:00 2024-02-20T02:06:22.135 INFO:teuthology.orchestra.run.smithi125.stdout:(11/32): python3-rgw-19.0.0-1579.g663dfa89.el9. 852 kB/s | 99 kB 00:00 2024-02-20T02:06:22.268 INFO:teuthology.orchestra.run.smithi125.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 795 kB/s | 106 kB 00:00 2024-02-20T02:06:22.328 INFO:teuthology.orchestra.run.smithi125.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-20T02:06:22.369 INFO:teuthology.orchestra.run.smithi125.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-20T02:06:22.452 INFO:teuthology.orchestra.run.smithi125.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-02-20T02:06:22.503 INFO:teuthology.orchestra.run.smithi125.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-20T02:06:22.570 INFO:teuthology.orchestra.run.smithi125.stdout:(17/32): librgw2-19.0.0-1579.g663dfa89.el9.x86_ 4.1 MB/s | 4.8 MB 00:01 2024-02-20T02:06:22.595 INFO:teuthology.orchestra.run.smithi125.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 448 kB/s | 41 kB 00:00 2024-02-20T02:06:22.628 INFO:teuthology.orchestra.run.smithi125.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 215 kB 00:00 2024-02-20T02:06:22.654 INFO:teuthology.orchestra.run.smithi125.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 576 kB/s | 34 kB 00:00 2024-02-20T02:06:22.687 INFO:teuthology.orchestra.run.smithi125.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 740 kB/s | 43 kB 00:00 2024-02-20T02:06:23.254 INFO:teuthology.orchestra.run.smithi125.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 514 kB/s | 309 kB 00:00 2024-02-20T02:06:23.421 INFO:teuthology.orchestra.run.smithi125.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-02-20T02:06:23.572 INFO:teuthology.orchestra.run.smithi125.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 324 kB/s | 49 kB 00:00 2024-02-20T02:06:23.730 INFO:teuthology.orchestra.run.smithi125.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-02-20T02:06:23.881 INFO:teuthology.orchestra.run.smithi125.stdout:(26/32): libarrow-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-02-20T02:06:23.989 INFO:teuthology.orchestra.run.smithi125.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-02-20T02:06:24.098 INFO:teuthology.orchestra.run.smithi125.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-02-20T02:06:24.373 INFO:teuthology.orchestra.run.smithi125.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-02-20T02:06:25.065 INFO:teuthology.orchestra.run.smithi125.stdout:(30/32): librbd1-19.0.0-1579.g663dfa89.el9.x86_ 4.5 MB/s | 3.1 MB 00:00 2024-02-20T02:06:25.149 INFO:teuthology.orchestra.run.smithi125.stdout:(31/32): librados2-19.0.0-1579.g663dfa89.el9.x8 3.0 MB/s | 3.2 MB 00:01 2024-02-20T02:06:30.398 INFO:teuthology.orchestra.run.smithi125.stdout:(32/32): ceph-common-19.0.0-1579.g663dfa89.el9. 1.7 MB/s | 18 MB 00:10 2024-02-20T02:06:30.401 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:06:30.401 INFO:teuthology.orchestra.run.smithi125.stdout:Total 4.9 MB/s | 55 MB 00:11 2024-02-20T02:06:30.402 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:06:30.438 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:06:30.438 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:06:30.902 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:06:30.902 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:06:32.356 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:06:32.705 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-20T02:06:32.791 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:32.882 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:32.973 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 3/34 2024-02-20T02:06:33.001 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:33.077 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:33.105 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-02-20T02:06:33.130 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-02-20T02:06:33.161 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-02-20T02:06:33.177 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-02-20T02:06:33.353 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 8/34 2024-02-20T02:06:33.384 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 8/34 2024-02-20T02:06:33.412 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-02-20T02:06:33.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9. 10/34 2024-02-20T02:06:33.484 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 11/34 2024-02-20T02:06:33.552 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 12/34 2024-02-20T02:06:33.574 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 13/34 2024-02-20T02:06:33.655 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 13/34 2024-02-20T02:06:33.708 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-02-20T02:06:33.819 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-20T02:06:33.867 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-20T02:06:33.907 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-20T02:06:33.931 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-20T02:06:33.962 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-20T02:06:33.996 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-20T02:06:34.328 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-20T02:06:34.358 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 22/34 2024-02-20T02:06:34.447 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 22/34 2024-02-20T02:06:34.479 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 23/34 2024-02-20T02:06:34.517 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-20T02:06:34.983 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-20T02:06:35.010 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 26/34 2024-02-20T02:06:35.081 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 26/34 2024-02-20T02:06:35.156 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 27/34 2024-02-20T02:06:35.187 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x8 28/34 2024-02-20T02:06:37.294 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:37.326 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:37.899 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 29/34 2024-02-20T02:06:37.937 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 30/34 2024-02-20T02:06:38.013 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 30/34 2024-02-20T02:06:38.013 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-20T02:06:38.013 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-20T02:06:38.013 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:38.054 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /sys 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /proc 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /mnt 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /var/tmp 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /home 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /root 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /tmp 2024-02-20T02:06:47.512 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:48.438 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 32/34 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 32/34 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-20T02:06:48.462 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:48.464 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-20T02:06:48.551 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-20T02:06:48.551 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:49.762 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 1/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 3/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 4/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 5/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_6 6/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 7/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9. 8/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x8 9/34 2024-02-20T02:06:49.763 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 10/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 11/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 12/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 13/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-20T02:06:49.764 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-20T02:06:49.765 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 31/34 2024-02-20T02:06:49.766 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-20T02:06:49.766 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 33/34 2024-02-20T02:06:49.890 INFO:teuthology.orchestra.run.smithi204.stdout:(9/9): ceph-test-19.0.0-1579.g663dfa89.el9.x86_ 4.1 MB/s | 122 MB 00:29 2024-02-20T02:06:49.892 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:06:49.893 INFO:teuthology.orchestra.run.smithi204.stdout:Total 4.1 MB/s | 124 MB 00:30 2024-02-20T02:06:49.893 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:06:49.914 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:06:49.914 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:06:50.763 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:06:50.763 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout:Upgraded: 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.874 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-20T02:06:50.875 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: re2-1:20211101-3.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:50.876 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:06:51.173 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-test 2024-02-20T02:06:51.742 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:00:35 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:06:51.909 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 122 M 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout: libasan x86_64 11.4.1-3.el9 CentOS-AppStream 417 k 2024-02-20T02:06:51.910 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 158 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: libubsan x86_64 11.4.1-3.el9 CentOS-AppStream 186 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout:Install 9 Packages 2024-02-20T02:06:51.911 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:06:51.912 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 124 M 2024-02-20T02:06:51.912 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.1 G 2024-02-20T02:06:51.912 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:06:51.920 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:06:51.990 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-02-20T02:06:52.033 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 2/9 2024-02-20T02:06:52.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 3/9 2024-02-20T02:06:52.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 4/9 2024-02-20T02:06:52.144 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 5/9 2024-02-20T02:06:52.168 INFO:teuthology.orchestra.run.smithi125.stdout:(1/9): libasan-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 417 kB 00:00 2024-02-20T02:06:52.175 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 6/9 2024-02-20T02:06:52.202 INFO:teuthology.orchestra.run.smithi125.stdout:(2/9): libubsan-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 186 kB 00:00 2024-02-20T02:06:52.217 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libasan-11.4.1-3.el9.x86_64 7/9 2024-02-20T02:06:52.244 INFO:teuthology.orchestra.run.smithi125.stdout:(3/9): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-02-20T02:06:52.245 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 8/9 2024-02-20T02:06:52.286 INFO:teuthology.orchestra.run.smithi125.stdout:(4/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-20T02:06:52.327 INFO:teuthology.orchestra.run.smithi125.stdout:(5/9): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-20T02:06:52.370 INFO:teuthology.orchestra.run.smithi125.stdout:(6/9): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-02-20T02:06:52.395 INFO:teuthology.orchestra.run.smithi125.stdout:(7/9): libcephsqlite-19.0.0-1579.g663dfa89.el9. 384 kB/s | 158 kB 00:00 2024-02-20T02:06:52.520 INFO:teuthology.orchestra.run.smithi125.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 810 kB/s | 122 kB 00:00 2024-02-20T02:06:57.328 INFO:teuthology.orchestra.run.smithi125.stdout:(9/9): ceph-test-19.0.0-1579.g663dfa89.el9.x86_ 23 MB/s | 122 MB 00:05 2024-02-20T02:06:57.331 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:06:57.331 INFO:teuthology.orchestra.run.smithi125.stdout:Total 23 MB/s | 124 MB 00:05 2024-02-20T02:06:57.332 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:06:57.355 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:06:57.355 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:06:58.181 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:06:58.182 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:06:59.355 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:06:59.420 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-02-20T02:06:59.455 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 2/9 2024-02-20T02:06:59.486 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : jq-1.6-16.el9.x86_64 3/9 2024-02-20T02:06:59.531 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 4/9 2024-02-20T02:06:59.570 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 5/9 2024-02-20T02:06:59.597 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 6/9 2024-02-20T02:06:59.638 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libasan-11.4.1-3.el9.x86_64 7/9 2024-02-20T02:06:59.666 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 8/9 2024-02-20T02:07:09.126 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 8/9 2024-02-20T02:07:09.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 9/9 2024-02-20T02:07:16.472 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 8/9 2024-02-20T02:07:16.546 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 9/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 9/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 1/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 2/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 3/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 4/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/9 2024-02-20T02:07:17.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 7/9 2024-02-20T02:07:17.673 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 8/9 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: libasan-11.4.1-3.el9.x86_64 2024-02-20T02:07:18.086 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: libubsan-11.4.1-3.el9.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:18.087 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:07:18.388 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph 2024-02-20T02:07:18.969 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:36 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:07:19.136 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================= 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================= 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 6.6 k 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 2.0 M 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 1.5 M 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 247 k 2024-02-20T02:07:19.139 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 4.9 M 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 118 M 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-20T02:07:19.140 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-20T02:07:19.141 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================= 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout:Install 37 Packages 2024-02-20T02:07:19.142 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:19.143 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 132 M 2024-02-20T02:07:19.143 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 1.4 G 2024-02-20T02:07:19.143 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:07:19.886 INFO:teuthology.orchestra.run.smithi204.stdout:(1/37): ceph-19.0.0-1579.g663dfa89.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-02-20T02:07:20.519 INFO:teuthology.orchestra.run.smithi204.stdout:(2/37): ceph-mon-19.0.0-1579.g663dfa89.el9.x86_ 7.7 MB/s | 4.9 MB 00:00 2024-02-20T02:07:20.728 INFO:teuthology.orchestra.run.smithi204.stdout:(3/37): ceph-mgr-19.0.0-1579.g663dfa89.el9.x86_ 1.5 MB/s | 1.5 MB 00:01 2024-02-20T02:07:20.853 INFO:teuthology.orchestra.run.smithi204.stdout:(4/37): ceph-mds-19.0.0-1579.g663dfa89.el9.x86_ 1.8 MB/s | 2.0 MB 00:01 2024-02-20T02:07:20.878 INFO:teuthology.orchestra.run.smithi204.stdout:(5/37): ceph-mgr-modules-core-19.0.0-1579.g663d 1.6 MB/s | 247 kB 00:00 2024-02-20T02:07:21.146 INFO:teuthology.orchestra.run.smithi204.stdout:(6/37): lua-5.4.4-4.el9.x86_64.rpm 640 kB/s | 187 kB 00:00 2024-02-20T02:07:21.254 INFO:teuthology.orchestra.run.smithi204.stdout:(7/37): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-02-20T02:07:21.288 INFO:teuthology.orchestra.run.smithi204.stdout:(8/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 144 kB 00:00 2024-02-20T02:07:21.321 INFO:teuthology.orchestra.run.smithi204.stdout:(9/37): python3-markupsafe-1.1.1-12.el9.x86_64. 471 kB/s | 32 kB 00:00 2024-02-20T02:07:21.347 INFO:teuthology.orchestra.run.smithi204.stdout:(10/37): python3-pytz-2021.1-5.el9.noarch.rpm 806 kB/s | 47 kB 00:00 2024-02-20T02:07:21.380 INFO:teuthology.orchestra.run.smithi204.stdout:(11/37): python3-toml-0.10.2-6.el9.noarch.rpm 657 kB/s | 39 kB 00:00 2024-02-20T02:07:21.548 INFO:teuthology.orchestra.run.smithi204.stdout:(12/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 241 kB 00:00 2024-02-20T02:07:21.906 INFO:teuthology.orchestra.run.smithi204.stdout:(13/37): python3-cryptography-36.0.1-4.el9.x86_ 2.2 MB/s | 1.1 MB 00:00 2024-02-20T02:07:21.931 INFO:teuthology.orchestra.run.smithi204.stdout:(14/37): python3-ply-3.11-14.el9.noarch.rpm 268 kB/s | 103 kB 00:00 2024-02-20T02:07:21.965 INFO:teuthology.orchestra.run.smithi204.stdout:(15/37): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-02-20T02:07:21.990 INFO:teuthology.orchestra.run.smithi204.stdout:(16/37): python3-requests-2.25.1-8.el9.noarch.r 1.9 MB/s | 113 kB 00:00 2024-02-20T02:07:22.024 INFO:teuthology.orchestra.run.smithi204.stdout:(17/37): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-02-20T02:07:22.149 INFO:teuthology.orchestra.run.smithi204.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-02-20T02:07:22.516 INFO:teuthology.orchestra.run.smithi204.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 118 kB/s | 43 kB 00:00 2024-02-20T02:07:22.542 INFO:teuthology.orchestra.run.smithi204.stdout:(20/37): luarocks-3.9.2-1.el9.noarch.rpm 292 kB/s | 151 kB 00:00 2024-02-20T02:07:22.700 INFO:teuthology.orchestra.run.smithi204.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 936 kB/s | 172 kB 00:00 2024-02-20T02:07:22.800 INFO:teuthology.orchestra.run.smithi204.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-02-20T02:07:22.917 INFO:teuthology.orchestra.run.smithi204.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 152 kB/s | 18 kB 00:00 2024-02-20T02:07:22.942 INFO:teuthology.orchestra.run.smithi204.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 895 kB/s | 358 kB 00:00 2024-02-20T02:07:23.018 INFO:teuthology.orchestra.run.smithi204.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-02-20T02:07:23.035 INFO:teuthology.orchestra.run.smithi204.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-02-20T02:07:23.110 INFO:teuthology.orchestra.run.smithi204.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-02-20T02:07:23.136 INFO:teuthology.orchestra.run.smithi204.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 461 kB/s | 46 kB 00:00 2024-02-20T02:07:23.236 INFO:teuthology.orchestra.run.smithi204.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 629 kB/s | 79 kB 00:00 2024-02-20T02:07:23.278 INFO:teuthology.orchestra.run.smithi204.stdout:(30/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-02-20T02:07:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-02-20T02:07:23.395 INFO:teuthology.orchestra.run.smithi204.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 769 kB/s | 90 kB 00:00 2024-02-20T02:07:23.445 INFO:teuthology.orchestra.run.smithi204.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 306 kB/s | 36 kB 00:00 2024-02-20T02:07:23.554 INFO:teuthology.orchestra.run.smithi204.stdout:(34/37): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-02-20T02:07:23.672 INFO:teuthology.orchestra.run.smithi204.stdout:(35/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-02-20T02:07:23.688 INFO:teuthology.orchestra.run.smithi204.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 149 kB/s | 20 kB 00:00 2024-02-20T02:07:24.743 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 9/9 2024-02-20T02:07:24.743 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 1/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 2/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 3/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 4/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : jq-1.6-16.el9.x86_64 7/9 2024-02-20T02:07:24.744 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 8/9 2024-02-20T02:07:25.169 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: jq-1.6-16.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: libasan-11.4.1-3.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: libubsan-11.4.1-3.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:25.170 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:07:25.472 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph 2024-02-20T02:07:26.047 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:10 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:07:26.215 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:07:26.218 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2024-02-20T02:07:26.218 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:07:26.218 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2024-02-20T02:07:26.218 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:07:26.218 INFO:teuthology.orchestra.run.smithi125.stdout: ceph x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 6.6 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 2.0 M 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 1.5 M 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 247 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 4.9 M 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 118 M 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-20T02:07:26.219 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-20T02:07:26.220 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-20T02:07:26.221 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout:========================================================================================= 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout:Install 37 Packages 2024-02-20T02:07:26.222 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:26.223 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 132 M 2024-02-20T02:07:26.223 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.4 G 2024-02-20T02:07:26.223 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:07:26.330 INFO:teuthology.orchestra.run.smithi204.stdout:(37/37): ceph-osd-19.0.0-1579.g663dfa89.el9.x86 20 MB/s | 118 MB 00:05 2024-02-20T02:07:26.334 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:07:26.335 INFO:teuthology.orchestra.run.smithi204.stdout:Total 18 MB/s | 132 MB 00:07 2024-02-20T02:07:26.373 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:07:26.415 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:07:26.415 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:07:26.704 INFO:teuthology.orchestra.run.smithi125.stdout:(1/37): ceph-19.0.0-1579.g663dfa89.el9.x86_64.r 30 kB/s | 6.6 kB 00:00 2024-02-20T02:07:26.813 INFO:teuthology.orchestra.run.smithi125.stdout:(2/37): ceph-mgr-19.0.0-1579.g663dfa89.el9.x86_ 4.5 MB/s | 1.5 MB 00:00 2024-02-20T02:07:27.205 INFO:teuthology.orchestra.run.smithi125.stdout:(3/37): ceph-mds-19.0.0-1579.g663dfa89.el9.x86_ 2.8 MB/s | 2.0 MB 00:00 2024-02-20T02:07:27.306 INFO:teuthology.orchestra.run.smithi125.stdout:(4/37): ceph-mgr-modules-core-19.0.0-1579.g663d 2.4 MB/s | 247 kB 00:00 2024-02-20T02:07:27.422 INFO:teuthology.orchestra.run.smithi125.stdout:(5/37): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-02-20T02:07:27.438 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:07:27.438 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:07:27.497 INFO:teuthology.orchestra.run.smithi125.stdout:(6/37): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-02-20T02:07:27.539 INFO:teuthology.orchestra.run.smithi125.stdout:(7/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-20T02:07:27.573 INFO:teuthology.orchestra.run.smithi125.stdout:(8/37): python3-markupsafe-1.1.1-12.el9.x86_64. 950 kB/s | 32 kB 00:00 2024-02-20T02:07:27.607 INFO:teuthology.orchestra.run.smithi125.stdout:(9/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-20T02:07:27.640 INFO:teuthology.orchestra.run.smithi125.stdout:(10/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-20T02:07:27.690 INFO:teuthology.orchestra.run.smithi125.stdout:(11/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-20T02:07:27.749 INFO:teuthology.orchestra.run.smithi125.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 20 MB/s | 1.1 MB 00:00 2024-02-20T02:07:27.783 INFO:teuthology.orchestra.run.smithi125.stdout:(13/37): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-02-20T02:07:27.816 INFO:teuthology.orchestra.run.smithi125.stdout:(14/37): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-02-20T02:07:27.850 INFO:teuthology.orchestra.run.smithi125.stdout:(15/37): python3-requests-2.25.1-8.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2024-02-20T02:07:27.883 INFO:teuthology.orchestra.run.smithi125.stdout:(16/37): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-02-20T02:07:27.992 INFO:teuthology.orchestra.run.smithi125.stdout:(17/37): ceph-mon-19.0.0-1579.g663dfa89.el9.x86 3.8 MB/s | 4.9 MB 00:01 2024-02-20T02:07:28.040 INFO:teuthology.orchestra.run.smithi125.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 143 kB/s | 22 kB 00:00 2024-02-20T02:07:28.301 INFO:teuthology.orchestra.run.smithi125.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 167 kB/s | 43 kB 00:00 2024-02-20T02:07:28.435 INFO:teuthology.orchestra.run.smithi125.stdout:(20/37): luarocks-3.9.2-1.el9.noarch.rpm 341 kB/s | 151 kB 00:00 2024-02-20T02:07:28.468 INFO:teuthology.orchestra.run.smithi125.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-02-20T02:07:28.527 INFO:teuthology.orchestra.run.smithi125.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-02-20T02:07:28.594 INFO:teuthology.orchestra.run.smithi125.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-02-20T02:07:28.611 INFO:teuthology.orchestra.run.smithi125.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-02-20T02:07:28.661 INFO:teuthology.orchestra.run.smithi125.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-02-20T02:07:28.678 INFO:teuthology.orchestra.run.smithi125.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 291 kB/s | 19 kB 00:00 2024-02-20T02:07:28.729 INFO:teuthology.orchestra.run.smithi125.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 292 kB/s | 20 kB 00:00 2024-02-20T02:07:28.745 INFO:teuthology.orchestra.run.smithi125.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 690 kB/s | 46 kB 00:00 2024-02-20T02:07:28.804 INFO:teuthology.orchestra.run.smithi125.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-02-20T02:07:28.863 INFO:teuthology.orchestra.run.smithi125.stdout:(30/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-02-20T02:07:28.879 INFO:teuthology.orchestra.run.smithi125.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 218 kB/s | 16 kB 00:00 2024-02-20T02:07:28.938 INFO:teuthology.orchestra.run.smithi125.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-02-20T02:07:28.963 INFO:teuthology.orchestra.run.smithi125.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 428 kB/s | 36 kB 00:00 2024-02-20T02:07:29.022 INFO:teuthology.orchestra.run.smithi125.stdout:(34/37): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-02-20T02:07:29.089 INFO:teuthology.orchestra.run.smithi125.stdout:(35/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-02-20T02:07:29.116 INFO:teuthology.orchestra.run.smithi125.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-02-20T02:07:29.336 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:07:29.391 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-02-20T02:07:29.424 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-02-20T02:07:29.451 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-02-20T02:07:29.538 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-02-20T02:07:29.575 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/37 2024-02-20T02:07:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-02-20T02:07:29.700 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-02-20T02:07:29.787 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-02-20T02:07:29.836 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/37 2024-02-20T02:07:29.907 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-02-20T02:07:29.961 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/37 2024-02-20T02:07:30.000 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/37 2024-02-20T02:07:30.049 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/37 2024-02-20T02:07:30.114 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/37 2024-02-20T02:07:30.309 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/37 2024-02-20T02:07:30.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/37 2024-02-20T02:07:30.393 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/37 2024-02-20T02:07:30.427 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/37 2024-02-20T02:07:30.457 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/37 2024-02-20T02:07:30.534 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/37 2024-02-20T02:07:30.576 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/37 2024-02-20T02:07:30.610 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/37 2024-02-20T02:07:30.650 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/37 2024-02-20T02:07:30.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/37 2024-02-20T02:07:30.720 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/37 2024-02-20T02:07:30.809 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/37 2024-02-20T02:07:30.851 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/37 2024-02-20T02:07:30.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/37 2024-02-20T02:07:31.060 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/37 2024-02-20T02:07:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/37 2024-02-20T02:07:31.354 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9. 31/37 2024-02-20T02:07:31.381 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 32/37 2024-02-20T02:07:31.406 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 32/37 2024-02-20T02:07:31.407 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:31.407 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-20T02:07:31.407 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-20T02:07:31.407 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-20T02:07:31.407 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:32.181 INFO:teuthology.orchestra.run.smithi125.stdout:(37/37): ceph-osd-19.0.0-1579.g663dfa89.el9.x86 22 MB/s | 118 MB 00:05 2024-02-20T02:07:32.186 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:07:32.186 INFO:teuthology.orchestra.run.smithi125.stdout:Total 22 MB/s | 132 MB 00:05 2024-02-20T02:07:32.209 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:07:32.252 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:07:32.252 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:07:33.258 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:07:33.259 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:07:35.168 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:07:35.233 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-02-20T02:07:35.266 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-02-20T02:07:35.294 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-02-20T02:07:35.365 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-02-20T02:07:35.409 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/37 2024-02-20T02:07:35.442 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-02-20T02:07:35.533 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-02-20T02:07:35.603 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-02-20T02:07:35.644 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/37 2024-02-20T02:07:35.717 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-02-20T02:07:35.770 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/37 2024-02-20T02:07:35.809 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/37 2024-02-20T02:07:35.858 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/37 2024-02-20T02:07:35.922 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/37 2024-02-20T02:07:36.106 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/37 2024-02-20T02:07:36.154 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/37 2024-02-20T02:07:36.193 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/37 2024-02-20T02:07:36.227 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/37 2024-02-20T02:07:36.257 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/37 2024-02-20T02:07:36.334 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/37 2024-02-20T02:07:36.367 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/37 2024-02-20T02:07:36.401 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/37 2024-02-20T02:07:36.441 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/37 2024-02-20T02:07:36.476 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/37 2024-02-20T02:07:36.511 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/37 2024-02-20T02:07:36.600 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/37 2024-02-20T02:07:36.642 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/37 2024-02-20T02:07:36.713 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/37 2024-02-20T02:07:36.852 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/37 2024-02-20T02:07:36.984 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/37 2024-02-20T02:07:37.143 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9. 31/37 2024-02-20T02:07:37.171 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 32/37 2024-02-20T02:07:37.196 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 32/37 2024-02-20T02:07:37.196 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:37.196 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-20T02:07:37.197 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-20T02:07:37.197 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-20T02:07:37.197 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:50.202 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 33/37 2024-02-20T02:07:50.239 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 34/37 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 34/37 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-20T02:07:50.266 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:50.732 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 35/37 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 35/37 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-20T02:07:50.757 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:50.981 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 36/37 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 36/37 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-20T02:07:51.003 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:07:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 37/37 2024-02-20T02:07:55.870 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 33/37 2024-02-20T02:07:55.906 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 34/37 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 34/37 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-20T02:07:55.932 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:56.965 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 35/37 2024-02-20T02:07:56.992 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 35/37 2024-02-20T02:07:56.992 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:56.992 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-20T02:07:56.992 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-20T02:07:56.992 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-20T02:07:56.993 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:57.416 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 36/37 2024-02-20T02:07:57.438 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 36/37 2024-02-20T02:07:57.439 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:07:57.439 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-20T02:07:57.439 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-20T02:07:57.439 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-20T02:07:57.439 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:07:57.752 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 37/37 2024-02-20T02:08:01.611 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 37/37 2024-02-20T02:08:01.611 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 1/37 2024-02-20T02:08:01.611 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 2/37 2024-02-20T02:08:01.611 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 3/37 2024-02-20T02:08:01.611 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 4/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 5/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9. 6/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 8/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 9/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 10/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 11/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 12/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 13/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 14/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 15/37 2024-02-20T02:08:01.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 16/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 18/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 20/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-02-20T02:08:01.613 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-02-20T02:08:01.614 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-02-20T02:08:01.614 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-02-20T02:08:01.614 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-02-20T02:08:01.615 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-02-20T02:08:01.615 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-02-20T02:08:01.615 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-02-20T02:08:01.615 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 35/37 2024-02-20T02:08:01.615 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:02.448 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-20T02:08:02.449 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-20T02:08:02.450 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-20T02:08:02.451 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:02.451 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:02.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-base 2024-02-20T02:08:03.304 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:21 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:03.418 INFO:teuthology.orchestra.run.smithi204.stdout:Package ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:03.468 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:03.469 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:03.469 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:03.531 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install cephadm 2024-02-20T02:08:04.114 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:22 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:04.281 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 748 k 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 748 k 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 754 k 2024-02-20T02:08:04.282 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:04.577 INFO:teuthology.orchestra.run.smithi204.stdout:cephadm-19.0.0-1579.g663dfa89.el9.noarch.rpm 2.5 MB/s | 748 kB 00:00 2024-02-20T02:08:04.578 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:04.578 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.5 MB/s | 748 kB 00:00 2024-02-20T02:08:04.578 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:04.583 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:04.584 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:04.589 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:04.589 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:04.667 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:05.069 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:05.165 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:05.622 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 37/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 1/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 2/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 3/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 4/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 5/37 2024-02-20T02:08:05.781 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9. 6/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 8/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 9/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 10/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 11/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 12/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 13/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 14/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ply-3.11-14.el9.noarch 15/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 16/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 18/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 20/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-02-20T02:08:05.782 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-02-20T02:08:05.783 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 35/37 2024-02-20T02:08:05.784 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:05.894 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:06.078 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-immutable-object-cache 2024-02-20T02:08:06.664 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:24 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:06.708 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-02-20T02:08:06.708 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:06.708 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:06.709 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: lua-5.4.4-4.el9.x86_64 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-20T02:08:06.710 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-20T02:08:06.711 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-20T02:08:06.712 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply-3.11-14.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-20T02:08:06.713 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-20T02:08:06.714 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-20T02:08:06.715 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:06.715 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:06.837 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 147 k 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:06.838 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:06.839 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 147 k 2024-02-20T02:08:06.839 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 481 k 2024-02-20T02:08:06.839 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:06.981 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-base 2024-02-20T02:08:07.192 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-immutable-object-cache-19.0.0-1579.g663dfa 417 kB/s | 147 kB 00:00 2024-02-20T02:08:07.193 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:07.193 INFO:teuthology.orchestra.run.smithi204.stdout:Total 415 kB/s | 147 kB 00:00 2024-02-20T02:08:07.193 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:07.200 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:07.200 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:07.252 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:07.252 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:07.389 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:07.452 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:07.476 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:07.477 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:08:07.477 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-20T02:08:07.477 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:07.557 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:51 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:07.671 INFO:teuthology.orchestra.run.smithi125.stdout:Package ceph-base-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:07.723 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:07.724 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:07.724 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:07.791 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install cephadm 2024-02-20T02:08:08.377 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:52 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:08.488 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:08.488 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:08.489 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:08.489 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:08.489 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:08.489 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:08.543 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:08.544 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:08.544 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:08.544 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:08.544 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:08.544 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 748 k 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 748 k 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 754 k 2024-02-20T02:08:08.545 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:08.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr 2024-02-20T02:08:08.981 INFO:teuthology.orchestra.run.smithi125.stdout:cephadm-19.0.0-1579.g663dfa89.el9.noarch.rpm 1.7 MB/s | 748 kB 00:00 2024-02-20T02:08:08.981 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:08.981 INFO:teuthology.orchestra.run.smithi125.stdout:Total 1.7 MB/s | 748 kB 00:00 2024-02-20T02:08:08.982 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:08.987 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:08.987 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:08.992 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:08.993 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:09.069 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:09.294 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:27 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:09.412 INFO:teuthology.orchestra.run.smithi204.stdout:Package ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:09.463 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:09.464 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:09.464 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:09.505 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:09.522 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-dashboard 2024-02-20T02:08:09.594 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:09.973 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:10.099 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:28 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:10.255 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/1 2024-02-20T02:08:10.256 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:10.256 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:10.256 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:10.256 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:10.256 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:10.265 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:10.266 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-02-20T02:08:10.266 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:10.266 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-02-20T02:08:10.267 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:10.267 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 1.8 M 2024-02-20T02:08:10.267 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:10.267 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-20T02:08:10.267 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 31 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 15 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-02-20T02:08:10.268 INFO:teuthology.orchestra.run.smithi204.stdout:Install 11 Packages 2024-02-20T02:08:10.269 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 5.8 M 2024-02-20T02:08:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 22 M 2024-02-20T02:08:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:10.451 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-immutable-object-cache 2024-02-20T02:08:10.723 INFO:teuthology.orchestra.run.smithi204.stdout:(1/11): ceph-prometheus-alerts-19.0.0-1579.g663 70 kB/s | 15 kB 00:00 2024-02-20T02:08:10.813 INFO:teuthology.orchestra.run.smithi204.stdout:(2/11): ceph-grafana-dashboards-19.0.0-1579.g66 101 kB/s | 31 kB 00:00 2024-02-20T02:08:10.913 INFO:teuthology.orchestra.run.smithi204.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 238 kB 00:00 2024-02-20T02:08:10.997 INFO:teuthology.orchestra.run.smithi204.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.6 MB/s | 863 kB 00:00 2024-02-20T02:08:11.031 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:55 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:11.156 INFO:teuthology.orchestra.run.smithi204.stdout:(5/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.2 MB/s | 548 kB 00:00 2024-02-20T02:08:11.172 INFO:teuthology.orchestra.run.smithi204.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 112 kB/s | 19 kB 00:00 2024-02-20T02:08:11.199 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:11.200 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:11.200 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2024-02-20T02:08:11.200 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:11.200 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:11.200 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 147 k 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:11.201 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 147 k 2024-02-20T02:08:11.202 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 481 k 2024-02-20T02:08:11.202 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:11.231 INFO:teuthology.orchestra.run.smithi204.stdout:(7/11): ceph-mgr-dashboard-19.0.0-1579.g663dfa8 2.4 MB/s | 1.8 MB 00:00 2024-02-20T02:08:11.390 INFO:teuthology.orchestra.run.smithi204.stdout:(8/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 8.7 MB/s | 2.0 MB 00:00 2024-02-20T02:08:11.407 INFO:teuthology.orchestra.run.smithi204.stdout:(9/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 617 kB/s | 144 kB 00:00 2024-02-20T02:08:11.424 INFO:teuthology.orchestra.run.smithi204.stdout:(10/11): python3-repoze-lru-0.7-16.el9.noarch.r 160 kB/s | 31 kB 00:00 2024-02-20T02:08:11.449 INFO:teuthology.orchestra.run.smithi204.stdout:(11/11): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-02-20T02:08:11.452 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:11.452 INFO:teuthology.orchestra.run.smithi204.stdout:Total 4.9 MB/s | 5.8 MB 00:01 2024-02-20T02:08:11.471 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:11.553 INFO:teuthology.orchestra.run.smithi125.stdout:ceph-immutable-object-cache-19.0.0-1579.g663dfa 417 kB/s | 147 kB 00:00 2024-02-20T02:08:11.553 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:11.554 INFO:teuthology.orchestra.run.smithi125.stdout:Total 415 kB/s | 147 kB 00:00 2024-02-20T02:08:11.554 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:11.561 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:11.561 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:11.613 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:11.613 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:11.662 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:11.663 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:11.749 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:11.804 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:11.829 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:11.829 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:08:11.829 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-20T02:08:11.829 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:12.143 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:12.229 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-02-20T02:08:12.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-02-20T02:08:12.370 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-02-20T02:08:12.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-02-20T02:08:12.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-02-20T02:08:12.638 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-02-20T02:08:12.699 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-02-20T02:08:12.728 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-02-20T02:08:12.759 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9 9/11 2024-02-20T02:08:12.844 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89. 1/1 2024-02-20T02:08:12.844 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:12.845 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:12.845 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:12.845 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:12.845 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:13.000 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el 10/11 2024-02-20T02:08:13.026 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 11/11 2024-02-20T02:08:13.055 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr 2024-02-20T02:08:13.639 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:57 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:13.752 INFO:teuthology.orchestra.run.smithi125.stdout:Package ceph-mgr-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:13.803 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:13.804 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:13.805 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:13.872 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-dashboard 2024-02-20T02:08:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 11/11 2024-02-20T02:08:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el 1/11 2024-02-20T02:08:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 2/11 2024-02-20T02:08:13.891 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9 3/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-02-20T02:08:13.892 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:14.256 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:14.257 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:14.448 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:01:58 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:14.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-20T02:08:14.613 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:14.614 INFO:teuthology.orchestra.run.smithi125.stdout:====================================================================================== 2024-02-20T02:08:14.614 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:14.614 INFO:teuthology.orchestra.run.smithi125.stdout:====================================================================================== 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 1.8 M 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 31 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 15 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:14.615 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:14.616 INFO:teuthology.orchestra.run.smithi125.stdout:====================================================================================== 2024-02-20T02:08:14.616 INFO:teuthology.orchestra.run.smithi125.stdout:Install 11 Packages 2024-02-20T02:08:14.616 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:14.616 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 5.8 M 2024-02-20T02:08:14.617 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 22 M 2024-02-20T02:08:14.617 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:15.070 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:33 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:15.240 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:15.241 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-02-20T02:08:15.241 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:15.241 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 7.4 M 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-20T02:08:15.242 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:Install 12 Packages 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 37 M 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 209 M 2024-02-20T02:08:15.243 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:15.348 INFO:teuthology.orchestra.run.smithi125.stdout:(1/11): ceph-grafana-dashboards-19.0.0-1579.g66 83 kB/s | 31 kB 00:00 2024-02-20T02:08:15.456 INFO:teuthology.orchestra.run.smithi125.stdout:(2/11): ceph-prometheus-alerts-19.0.0-1579.g663 31 kB/s | 15 kB 00:00 2024-02-20T02:08:15.464 INFO:teuthology.orchestra.run.smithi204.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 297 kB/s | 32 kB 00:00 2024-02-20T02:08:15.481 INFO:teuthology.orchestra.run.smithi125.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.7 MB/s | 238 kB 00:00 2024-02-20T02:08:15.514 INFO:teuthology.orchestra.run.smithi204.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 645 kB/s | 17 kB 00:00 2024-02-20T02:08:15.540 INFO:teuthology.orchestra.run.smithi204.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-02-20T02:08:15.644 INFO:teuthology.orchestra.run.smithi125.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.5 MB/s | 863 kB 00:00 2024-02-20T02:08:15.707 INFO:teuthology.orchestra.run.smithi204.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2024-02-20T02:08:15.840 INFO:teuthology.orchestra.run.smithi204.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-02-20T02:08:15.891 INFO:teuthology.orchestra.run.smithi204.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.6 MB/s | 3.0 MB 00:00 2024-02-20T02:08:15.898 INFO:teuthology.orchestra.run.smithi125.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 76 kB/s | 19 kB 00:00 2024-02-20T02:08:15.916 INFO:teuthology.orchestra.run.smithi204.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 217 kB 00:00 2024-02-20T02:08:16.074 INFO:teuthology.orchestra.run.smithi125.stdout:(6/11): ceph-mgr-dashboard-19.0.0-1579.g663dfa8 1.6 MB/s | 1.8 MB 00:01 2024-02-20T02:08:16.100 INFO:teuthology.orchestra.run.smithi204.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 802 kB 00:00 2024-02-20T02:08:16.258 INFO:teuthology.orchestra.run.smithi204.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-02-20T02:08:16.283 INFO:teuthology.orchestra.run.smithi204.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 194 kB 00:00 2024-02-20T02:08:16.417 INFO:teuthology.orchestra.run.smithi204.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-02-20T02:08:16.533 INFO:teuthology.orchestra.run.smithi125.stdout:(7/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 521 kB/s | 548 kB 00:01 2024-02-20T02:08:16.558 INFO:teuthology.orchestra.run.smithi125.stdout:(8/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 299 kB/s | 144 kB 00:00 2024-02-20T02:08:16.675 INFO:teuthology.orchestra.run.smithi125.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 217 kB/s | 31 kB 00:00 2024-02-20T02:08:16.742 INFO:teuthology.orchestra.run.smithi125.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 1.0 MB/s | 188 kB 00:00 2024-02-20T02:08:17.683 INFO:teuthology.orchestra.run.smithi125.stdout:(11/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 1.1 MB/s | 2.0 MB 00:01 2024-02-20T02:08:17.687 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:17.687 INFO:teuthology.orchestra.run.smithi125.stdout:Total 1.9 MB/s | 5.8 MB 00:03 2024-02-20T02:08:17.707 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:17.725 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:17.725 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:17.899 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:17.900 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:18.309 INFO:teuthology.orchestra.run.smithi204.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 2.5 MB/s | 7.4 MB 00:02 2024-02-20T02:08:18.312 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:18.312 INFO:teuthology.orchestra.run.smithi204.stdout:Total 12 MB/s | 37 MB 00:03 2024-02-20T02:08:18.326 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:18.353 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:18.353 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:18.364 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:18.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-02-20T02:08:18.495 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-02-20T02:08:18.586 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-02-20T02:08:18.711 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:18.711 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:18.712 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-02-20T02:08:18.757 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-02-20T02:08:18.848 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-02-20T02:08:18.901 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-02-20T02:08:18.929 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-02-20T02:08:18.960 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9 9/11 2024-02-20T02:08:19.210 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el 10/11 2024-02-20T02:08:19.244 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 11/11 2024-02-20T02:08:19.419 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:19.498 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-20T02:08:19.530 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-20T02:08:19.630 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-20T02:08:19.684 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-20T02:08:19.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-20T02:08:19.891 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-20T02:08:19.968 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 11/11 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el 1/11 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noa 2/11 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9 3/11 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-02-20T02:08:20.032 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-02-20T02:08:20.033 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-02-20T02:08:20.033 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-02-20T02:08:20.033 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-02-20T02:08:20.033 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-02-20T02:08:20.033 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-02-20T02:08:20.377 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-02-20T02:08:20.377 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:20.378 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:20.637 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-20T02:08:20.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-20T02:08:20.752 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-20T02:08:21.208 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:05 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:21.375 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:21.376 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 7.4 M 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-20T02:08:21.377 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout:Install 12 Packages 2024-02-20T02:08:21.378 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:21.379 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 37 M 2024-02-20T02:08:21.379 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 209 M 2024-02-20T02:08:21.379 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi125.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 218 kB/s | 32 kB 00:00 2024-02-20T02:08:21.673 INFO:teuthology.orchestra.run.smithi125.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 537 kB/s | 17 kB 00:00 2024-02-20T02:08:21.706 INFO:teuthology.orchestra.run.smithi125.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.1 MB/s | 34 kB 00:00 2024-02-20T02:08:21.757 INFO:teuthology.orchestra.run.smithi125.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-02-20T02:08:21.932 INFO:teuthology.orchestra.run.smithi125.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-02-20T02:08:21.978 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-20T02:08:22.007 INFO:teuthology.orchestra.run.smithi125.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-20T02:08:22.033 INFO:teuthology.orchestra.run.smithi125.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-02-20T02:08:22.166 INFO:teuthology.orchestra.run.smithi125.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 6.0 MB/s | 802 kB 00:00 2024-02-20T02:08:22.333 INFO:teuthology.orchestra.run.smithi125.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-02-20T02:08:22.359 INFO:teuthology.orchestra.run.smithi125.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 194 kB 00:00 2024-02-20T02:08:22.476 INFO:teuthology.orchestra.run.smithi125.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 7.6 MB/s | 7.4 MB 00:00 2024-02-20T02:08:22.609 INFO:teuthology.orchestra.run.smithi125.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 892 kB/s | 245 kB 00:00 2024-02-20T02:08:22.612 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:22.612 INFO:teuthology.orchestra.run.smithi125.stdout:Total 30 MB/s | 37 MB 00:01 2024-02-20T02:08:22.625 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:22.650 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:22.650 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:23.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-20T02:08:23.012 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:23.013 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:23.041 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 12/12 2024-02-20T02:08:23.722 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:23.799 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-20T02:08:23.830 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-20T02:08:23.924 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-20T02:08:23.975 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-20T02:08:24.153 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-20T02:08:24.182 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-20T02:08:24.259 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-20T02:08:24.915 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-20T02:08:25.028 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 12/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 1/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-20T02:08:25.594 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-20T02:08:25.595 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-20T02:08:25.595 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-20T02:08:25.595 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-20T02:08:25.595 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-20T02:08:25.595 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-20T02:08:26.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-20T02:08:26.051 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:26.051 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:26.052 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:26.259 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-20T02:08:26.530 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-rook 2024-02-20T02:08:27.124 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:45 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:27.290 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-20T02:08:27.290 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:27.291 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:27.291 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 52 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-20T02:08:27.292 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout:Install 13 Packages 2024-02-20T02:08:27.293 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:27.294 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 2.0 M 2024-02-20T02:08:27.294 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 26 M 2024-02-20T02:08:27.294 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:27.314 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 12/12 2024-02-20T02:08:27.601 INFO:teuthology.orchestra.run.smithi204.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 197 kB/s | 24 kB 00:00 2024-02-20T02:08:27.626 INFO:teuthology.orchestra.run.smithi204.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 121 kB/s | 18 kB 00:00 2024-02-20T02:08:27.685 INFO:teuthology.orchestra.run.smithi204.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.0 MB/s | 174 kB 00:00 2024-02-20T02:08:27.710 INFO:teuthology.orchestra.run.smithi204.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-02-20T02:08:27.744 INFO:teuthology.orchestra.run.smithi204.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-02-20T02:08:27.769 INFO:teuthology.orchestra.run.smithi204.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 729 kB/s | 43 kB 00:00 2024-02-20T02:08:27.836 INFO:teuthology.orchestra.run.smithi204.stdout:(7/13): ceph-mgr-rook-19.0.0-1579.g663dfa89.el9 145 kB/s | 52 kB 00:00 2024-02-20T02:08:27.870 INFO:teuthology.orchestra.run.smithi204.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 149 kB/s | 15 kB 00:00 2024-02-20T02:08:27.895 INFO:teuthology.orchestra.run.smithi204.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 213 kB/s | 32 kB 00:00 2024-02-20T02:08:27.979 INFO:teuthology.orchestra.run.smithi204.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-02-20T02:08:28.021 INFO:teuthology.orchestra.run.smithi204.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 1.1 MB/s | 203 kB 00:00 2024-02-20T02:08:28.054 INFO:teuthology.orchestra.run.smithi204.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-02-20T02:08:28.113 INFO:teuthology.orchestra.run.smithi204.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.2 MB/s | 1.0 MB 00:00 2024-02-20T02:08:28.115 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:28.115 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2024-02-20T02:08:28.115 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:28.128 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:28.128 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:28.228 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:28.228 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:28.752 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:28.910 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-20T02:08:28.956 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-20T02:08:29.020 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-20T02:08:29.057 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-20T02:08:29.099 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-20T02:08:29.191 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-20T02:08:29.292 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-20T02:08:29.342 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-20T02:08:29.911 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 12/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1579.g663df 1/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-20T02:08:29.912 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-20T02:08:30.073 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-20T02:08:30.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-20T02:08:30.164 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-20T02:08:30.213 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-20T02:08:30.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 13/13 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-20T02:08:30.366 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:30.367 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:30.755 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-rook 2024-02-20T02:08:31.010 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 13/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 1/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-20T02:08:31.011 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-20T02:08:31.348 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:15 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:31.386 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-20T02:08:31.386 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:31.386 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:31.386 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:31.386 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:31.387 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:31.518 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 52 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-20T02:08:31.519 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout:Install 13 Packages 2024-02-20T02:08:31.520 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:31.521 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 2.0 M 2024-02-20T02:08:31.521 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 26 M 2024-02-20T02:08:31.521 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:31.627 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-cephadm 2024-02-20T02:08:31.715 INFO:teuthology.orchestra.run.smithi125.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 207 kB/s | 24 kB 00:00 2024-02-20T02:08:31.741 INFO:teuthology.orchestra.run.smithi125.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 126 kB/s | 18 kB 00:00 2024-02-20T02:08:31.791 INFO:teuthology.orchestra.run.smithi125.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-20T02:08:31.808 INFO:teuthology.orchestra.run.smithi125.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-02-20T02:08:31.833 INFO:teuthology.orchestra.run.smithi125.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.9 MB/s | 211 kB 00:00 2024-02-20T02:08:31.858 INFO:teuthology.orchestra.run.smithi125.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 857 kB/s | 43 kB 00:00 2024-02-20T02:08:31.926 INFO:teuthology.orchestra.run.smithi125.stdout:(7/13): ceph-mgr-rook-19.0.0-1579.g663dfa89.el9 159 kB/s | 52 kB 00:00 2024-02-20T02:08:32.209 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:50 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:32.376 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:32.376 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:32.376 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:32.376 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 138 k 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================== 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout:Install 6 Packages 2024-02-20T02:08:32.377 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:32.378 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 6.8 M 2024-02-20T02:08:32.378 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 33 M 2024-02-20T02:08:32.378 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:32.567 INFO:teuthology.orchestra.run.smithi125.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 21 kB/s | 15 kB 00:00 2024-02-20T02:08:32.681 INFO:teuthology.orchestra.run.smithi204.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 227 kB 00:00 2024-02-20T02:08:32.782 INFO:teuthology.orchestra.run.smithi204.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1579.g663dfa89.e 540 kB/s | 138 kB 00:00 2024-02-20T02:08:32.818 INFO:teuthology.orchestra.run.smithi125.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 33 kB/s | 32 kB 00:00 2024-02-20T02:08:32.866 INFO:teuthology.orchestra.run.smithi204.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-02-20T02:08:33.358 INFO:teuthology.orchestra.run.smithi204.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 100 kB/s | 58 kB 00:00 2024-02-20T02:08:33.441 INFO:teuthology.orchestra.run.smithi204.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 89 kB/s | 51 kB 00:00 2024-02-20T02:08:33.551 INFO:teuthology.orchestra.run.smithi125.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 125 kB/s | 203 kB 00:01 2024-02-20T02:08:34.184 INFO:teuthology.orchestra.run.smithi125.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 646 kB/s | 1.0 MB 00:01 2024-02-20T02:08:34.218 INFO:teuthology.orchestra.run.smithi125.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 42 kB/s | 59 kB 00:01 2024-02-20T02:08:34.235 INFO:teuthology.orchestra.run.smithi125.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 131 kB/s | 90 kB 00:00 2024-02-20T02:08:34.241 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:34.241 INFO:teuthology.orchestra.run.smithi125.stdout:Total 771 kB/s | 2.0 MB 00:02 2024-02-20T02:08:34.242 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:34.256 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:34.256 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:34.359 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:34.360 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:34.878 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:35.060 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-20T02:08:35.111 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-20T02:08:35.166 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-20T02:08:35.212 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-20T02:08:35.253 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-20T02:08:35.348 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-20T02:08:35.366 INFO:teuthology.orchestra.run.smithi204.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 204 kB/s | 548 kB 00:02 2024-02-20T02:08:35.368 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:35.368 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.3 MB/s | 6.8 MB 00:02 2024-02-20T02:08:35.369 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:35.379 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:35.379 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:35.456 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-20T02:08:35.476 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:35.476 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:35.513 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-20T02:08:35.784 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:35.956 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-20T02:08:36.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-20T02:08:36.245 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-20T02:08:36.294 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-20T02:08:36.328 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-20T02:08:36.375 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-20T02:08:36.397 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 13/13 2024-02-20T02:08:36.526 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-20T02:08:36.614 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-20T02:08:36.684 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-20T02:08:36.707 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 6/6 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 13/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 1/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-20T02:08:37.206 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-20T02:08:37.207 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-20T02:08:37.518 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 6/6 2024-02-20T02:08:37.519 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/6 2024-02-20T02:08:37.519 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-20T02:08:37.519 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-20T02:08:37.519 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-20T02:08:37.519 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-20T02:08:37.591 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:37.592 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:37.823 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-mgr-cephadm 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-20T02:08:37.844 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-20T02:08:37.845 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:37.845 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:38.112 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-fuse 2024-02-20T02:08:38.408 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:22 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:38.580 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:38.580 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:38.580 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:38.580 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 138 k 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:======================================================================================== 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Install 6 Packages 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 6.8 M 2024-02-20T02:08:38.581 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 33 M 2024-02-20T02:08:38.582 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:38.693 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:56 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 858 k 2024-02-20T02:08:38.861 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout:Install 2 Packages 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:38.862 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 937 k 2024-02-20T02:08:38.863 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 3.1 M 2024-02-20T02:08:38.863 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:38.995 INFO:teuthology.orchestra.run.smithi204.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 595 kB/s | 79 kB 00:00 2024-02-20T02:08:39.119 INFO:teuthology.orchestra.run.smithi125.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 227 kB 00:00 2024-02-20T02:08:39.195 INFO:teuthology.orchestra.run.smithi125.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1579.g663dfa89.e 542 kB/s | 138 kB 00:00 2024-02-20T02:08:39.278 INFO:teuthology.orchestra.run.smithi125.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-02-20T02:08:39.337 INFO:teuthology.orchestra.run.smithi204.stdout:(2/2): ceph-fuse-19.0.0-1579.g663dfa89.el9.x86_ 1.8 MB/s | 858 kB 00:00 2024-02-20T02:08:39.338 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:39.338 INFO:teuthology.orchestra.run.smithi204.stdout:Total 1.9 MB/s | 937 kB 00:00 2024-02-20T02:08:39.338 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:39.358 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:39.358 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:39.429 INFO:teuthology.orchestra.run.smithi125.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 249 kB/s | 58 kB 00:00 2024-02-20T02:08:39.435 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:39.435 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:39.462 INFO:teuthology.orchestra.run.smithi125.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-02-20T02:08:39.495 INFO:teuthology.orchestra.run.smithi125.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-02-20T02:08:39.497 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:39.498 INFO:teuthology.orchestra.run.smithi125.stdout:Total 7.4 MB/s | 6.8 MB 00:00 2024-02-20T02:08:39.498 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:39.507 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:39.507 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:39.597 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:39.598 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:39.613 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:39.775 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-20T02:08:39.853 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2/2 2024-02-20T02:08:39.906 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:40.069 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-20T02:08:40.114 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-20T02:08:40.612 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2/2 2024-02-20T02:08:40.612 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/2 2024-02-20T02:08:40.624 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-20T02:08:40.708 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-20T02:08:40.769 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-20T02:08:40.793 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 6/6 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:40.943 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:41.209 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-volume 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 6/6 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 1/6 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-20T02:08:41.580 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-20T02:08:41.809 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:02:59 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:41.890 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-20T02:08:41.890 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:41.890 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:41.891 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:41.977 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 264 k 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout:Install 2 Packages 2024-02-20T02:08:41.978 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:41.979 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 332 k 2024-02-20T02:08:41.979 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 1.5 M 2024-02-20T02:08:41.979 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:42.100 INFO:teuthology.orchestra.run.smithi204.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 569 kB/s | 69 kB 00:00 2024-02-20T02:08:42.158 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-fuse 2024-02-20T02:08:42.375 INFO:teuthology.orchestra.run.smithi204.stdout:(2/2): ceph-volume-19.0.0-1579.g663dfa89.el9.no 666 kB/s | 264 kB 00:00 2024-02-20T02:08:42.376 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:42.376 INFO:teuthology.orchestra.run.smithi204.stdout:Total 835 kB/s | 332 kB 00:00 2024-02-20T02:08:42.377 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:42.385 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:42.385 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:42.426 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:42.426 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:42.736 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:26 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:42.826 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-20T02:08:42.853 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2/2 2024-02-20T02:08:42.869 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2/2 2024-02-20T02:08:42.869 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:08:42.869 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-20T02:08:42.869 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:42.900 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 858 k 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-20T02:08:42.901 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:Install 2 Packages 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 937 k 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 3.1 M 2024-02-20T02:08:42.902 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:43.022 INFO:teuthology.orchestra.run.smithi125.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 657 kB/s | 79 kB 00:00 2024-02-20T02:08:43.364 INFO:teuthology.orchestra.run.smithi125.stdout:(2/2): ceph-fuse-19.0.0-1579.g663dfa89.el9.x86_ 1.8 MB/s | 858 kB 00:00 2024-02-20T02:08:43.365 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:43.366 INFO:teuthology.orchestra.run.smithi125.stdout:Total 2.0 MB/s | 937 kB 00:00 2024-02-20T02:08:43.366 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:43.386 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:43.386 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:43.466 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:43.466 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:43.616 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 1/2 2024-02-20T02:08:43.649 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:43.807 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-20T02:08:43.880 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2/2 2024-02-20T02:08:43.950 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-20T02:08:43.950 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:43.951 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:43.951 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:43.951 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-20T02:08:43.951 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:43.951 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:44.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librados-devel 2024-02-20T02:08:44.685 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2/2 2024-02-20T02:08:44.685 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/2 2024-02-20T02:08:44.861 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:02 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:44.990 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-20T02:08:44.990 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:44.990 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:44.991 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-20T02:08:44.991 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:44.991 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:45.027 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:45.027 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 127 k 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:45.028 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:45.029 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 127 k 2024-02-20T02:08:45.029 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 457 k 2024-02-20T02:08:45.029 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:45.238 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-volume 2024-02-20T02:08:45.415 INFO:teuthology.orchestra.run.smithi204.stdout:librados-devel-19.0.0-1579.g663dfa89.el9.x86_64 328 kB/s | 127 kB 00:00 2024-02-20T02:08:45.416 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:45.416 INFO:teuthology.orchestra.run.smithi204.stdout:Total 327 kB/s | 127 kB 00:00 2024-02-20T02:08:45.416 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:45.422 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:45.422 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:45.483 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:45.483 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:45.641 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:45.748 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:45.832 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:29 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:46.001 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:46.001 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume noarch 2:19.0.0-1579.g663dfa89.el9 ceph-noarch 264 k 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Install 2 Packages 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 332 k 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 1.5 M 2024-02-20T02:08:46.002 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:46.128 INFO:teuthology.orchestra.run.smithi125.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 551 kB/s | 69 kB 00:00 2024-02-20T02:08:46.183 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:46.444 INFO:teuthology.orchestra.run.smithi125.stdout:(2/2): ceph-volume-19.0.0-1579.g663dfa89.el9.no 597 kB/s | 264 kB 00:00 2024-02-20T02:08:46.446 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:46.446 INFO:teuthology.orchestra.run.smithi125.stdout:Total 749 kB/s | 332 kB 00:00 2024-02-20T02:08:46.446 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:46.454 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:46.455 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:46.499 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:46.500 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:46.503 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:46.503 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:46.503 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:46.503 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:46.504 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:46.504 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:46.707 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:46.753 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install libcephfs2 2024-02-20T02:08:46.880 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-20T02:08:46.914 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2/2 2024-02-20T02:08:46.929 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2/2 2024-02-20T02:08:46.929 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:08:46.929 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-20T02:08:46.929 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:47.346 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:05 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:47.463 INFO:teuthology.orchestra.run.smithi204.stdout:Package libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:47.514 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:47.516 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:47.516 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:47.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install libcephfs-devel 2024-02-20T02:08:47.650 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 1/2 2024-02-20T02:08:48.015 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-20T02:08:48.015 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:48.016 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:48.016 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume-2:19.0.0-1579.g663dfa89.el9.noarch 2024-02-20T02:08:48.016 INFO:teuthology.orchestra.run.smithi125.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-20T02:08:48.016 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:48.016 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:48.151 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:06 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:48.310 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librados-devel 2024-02-20T02:08:48.317 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 31 k 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:48.318 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:48.319 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 31 k 2024-02-20T02:08:48.319 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 140 k 2024-02-20T02:08:48.319 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:48.560 INFO:teuthology.orchestra.run.smithi204.stdout:libcephfs-devel-19.0.0-1579.g663dfa89.el9.x86_6 129 kB/s | 31 kB 00:00 2024-02-20T02:08:48.561 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:48.561 INFO:teuthology.orchestra.run.smithi204.stdout:Total 128 kB/s | 31 kB 00:00 2024-02-20T02:08:48.561 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:48.565 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:48.565 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:48.594 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:48.595 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:48.687 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:48.800 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:48.894 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:32 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:49.059 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 127 k 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 127 k 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 457 k 2024-02-20T02:08:49.060 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:49.176 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:49.292 INFO:teuthology.orchestra.run.smithi125.stdout:librados-devel-19.0.0-1579.g663dfa89.el9.x86_64 548 kB/s | 127 kB 00:00 2024-02-20T02:08:49.292 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:49.293 INFO:teuthology.orchestra.run.smithi125.stdout:Total 545 kB/s | 127 kB 00:00 2024-02-20T02:08:49.293 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:49.298 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:49.298 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:49.358 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:49.358 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:49.467 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:49.467 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:49.468 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:49.468 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:49.468 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:49.468 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:49.517 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:49.606 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:49.646 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librados2 2024-02-20T02:08:50.089 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:50.231 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:08 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:50.345 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:50.396 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:50.398 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:50.398 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:50.407 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:50.407 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:50.408 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:50.408 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:50.408 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:50.408 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:50.458 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librbd1 2024-02-20T02:08:50.661 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install libcephfs2 2024-02-20T02:08:51.042 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:09 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:51.157 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:51.208 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:51.209 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:51.209 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:51.256 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:35 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:51.262 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rados 2024-02-20T02:08:51.375 INFO:teuthology.orchestra.run.smithi125.stdout:Package libcephfs2-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:51.426 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:51.427 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:51.427 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:51.488 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install libcephfs-devel 2024-02-20T02:08:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:09 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:51.964 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:52.016 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:52.017 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:52.017 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:52.063 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:36 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:52.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rgw 2024-02-20T02:08:52.228 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 31 k 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:52.229 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 31 k 2024-02-20T02:08:52.230 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 140 k 2024-02-20T02:08:52.230 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:52.395 INFO:teuthology.orchestra.run.smithi125.stdout:libcephfs-devel-19.0.0-1579.g663dfa89.el9.x86_6 188 kB/s | 31 kB 00:00 2024-02-20T02:08:52.396 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:52.396 INFO:teuthology.orchestra.run.smithi125.stdout:Total 186 kB/s | 31 kB 00:00 2024-02-20T02:08:52.396 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:52.400 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:52.400 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:52.427 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:52.427 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:52.521 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:52.627 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:52.661 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:10 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:52.776 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:52.826 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:52.827 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:52.827 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:52.894 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-cephfs 2024-02-20T02:08:53.009 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:53.311 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:53.311 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:53.311 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:08:53.312 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:53.312 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:53.312 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:53.472 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:11 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:53.504 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librados2 2024-02-20T02:08:53.586 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:53.638 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:53.639 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:53.639 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:53.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rbd 2024-02-20T02:08:54.079 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:38 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:54.193 INFO:teuthology.orchestra.run.smithi125.stdout:Package librados2-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:54.244 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:54.245 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:54.245 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:54.307 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install librbd1 2024-02-20T02:08:54.325 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:12 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:54.445 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:54.499 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:54.500 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-02-20T02:08:54.500 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:54.556 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-fuse 2024-02-20T02:08:54.883 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:38 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:54.996 INFO:teuthology.orchestra.run.smithi125.stdout:Package librbd1-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:55.046 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:55.047 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:55.047 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:55.104 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rados 2024-02-20T02:08:55.160 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:13 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:55.325 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:55.326 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:55.326 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-02-20T02:08:55.326 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:55.326 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 86 k 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 86 k 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 256 k 2024-02-20T02:08:55.327 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:55.685 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:39 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:55.799 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rados-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:55.831 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-fuse-19.0.0-1579.g663dfa89.el9.x86_64.rpm 171 kB/s | 86 kB 00:00 2024-02-20T02:08:55.831 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:55.832 INFO:teuthology.orchestra.run.smithi204.stdout:Total 170 kB/s | 86 kB 00:00 2024-02-20T02:08:55.832 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:55.839 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:55.839 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:55.850 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:55.851 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:55.852 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:55.895 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:55.895 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:55.919 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rgw 2024-02-20T02:08:56.039 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:56.147 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:56.496 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:40 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:56.577 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:56.609 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rgw-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:56.660 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:56.661 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:56.662 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:56.736 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-cephfs 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:56.898 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:08:57.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-mirror 2024-02-20T02:08:57.307 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:41 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:57.420 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-cephfs-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:57.472 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:57.473 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:57.473 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:57.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install python3-rbd 2024-02-20T02:08:57.713 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:15 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:08:57.884 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:08:57.884 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.0 M 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 3.0 M 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 14 M 2024-02-20T02:08:57.885 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:08:58.151 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:42 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:58.270 INFO:teuthology.orchestra.run.smithi125.stdout:Package python3-rbd-2:19.0.0-1579.g663dfa89.el9.x86_64 is already installed. 2024-02-20T02:08:58.323 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:58.324 INFO:teuthology.orchestra.run.smithi125.stdout:Nothing to do. 2024-02-20T02:08:58.324 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:08:58.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-fuse 2024-02-20T02:08:58.986 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:42 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:08:59.151 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout: Package Architecture Version Repository Size 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 86 k 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:08:59.152 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:08:59.153 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:08:59.153 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:08:59.153 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 86 k 2024-02-20T02:08:59.153 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 256 k 2024-02-20T02:08:59.153 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:08:59.376 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-mirror-19.0.0-1579.g663dfa89.el9.x86_64.rpm 2.0 MB/s | 3.0 MB 00:01 2024-02-20T02:08:59.377 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:59.377 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.0 MB/s | 3.0 MB 00:01 2024-02-20T02:08:59.378 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:08:59.388 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:08:59.389 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:08:59.467 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:08:59.476 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:08:59.665 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-fuse-19.0.0-1579.g663dfa89.el9.x86_64.rpm 168 kB/s | 86 kB 00:00 2024-02-20T02:08:59.666 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:08:59.666 INFO:teuthology.orchestra.run.smithi125.stdout:Total 167 kB/s | 86 kB 00:00 2024-02-20T02:08:59.666 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:08:59.673 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:08:59.673 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:08:59.729 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:08:59.729 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:08:59.866 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:08:59.899 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:08:59.960 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:59.972 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-20T02:08:59.986 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:00.417 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:00.711 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:09:00.934 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-mirror 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:01.175 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:09:01.434 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-nbd 2024-02-20T02:09:01.522 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:45 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:09:01.693 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repo Size 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 3.0 M 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:01.694 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 3.0 M 2024-02-20T02:09:01.695 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 14 M 2024-02-20T02:09:01.695 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:09:02.021 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:03:20 ago on Tue 20 Feb 2024 02:05:42 AM UTC. 2024-02-20T02:09:02.060 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-mirror-19.0.0-1579.g663dfa89.el9.x86_64.rpm 8.3 MB/s | 3.0 MB 00:00 2024-02-20T02:09:02.061 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:09:02.061 INFO:teuthology.orchestra.run.smithi125.stdout:Total 8.3 MB/s | 3.0 MB 00:00 2024-02-20T02:09:02.061 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:09:02.070 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:09:02.070 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:09:02.142 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:09:02.143 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:09:02.184 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 169 k 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-02-20T02:09:02.185 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-02-20T02:09:02.186 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-02-20T02:09:02.186 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:02.186 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 169 k 2024-02-20T02:09:02.186 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 540 k 2024-02-20T02:09:02.186 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-02-20T02:09:02.439 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-nbd-19.0.0-1579.g663dfa89.el9.x86_64.rpm 669 kB/s | 169 kB 00:00 2024-02-20T02:09:02.440 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:09:02.440 INFO:teuthology.orchestra.run.smithi204.stdout:Total 665 kB/s | 169 kB 00:00 2024-02-20T02:09:02.440 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-02-20T02:09:02.447 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-02-20T02:09:02.447 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-02-20T02:09:02.508 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-02-20T02:09:02.509 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-02-20T02:09:02.574 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:09:02.636 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:02.659 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-20T02:09:02.661 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:02.770 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:03.227 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:03.523 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-02-20T02:09:03.761 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:03.846 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:09:04.100 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install rbd-nbd 2024-02-20T02:09:04.692 INFO:teuthology.orchestra.run.smithi125.stdout:Last metadata expiration check: 0:02:48 ago on Tue 20 Feb 2024 02:06:16 AM UTC. 2024-02-20T02:09:04.857 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout: Package Architecture Version Repository Size 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd x86_64 2:19.0.0-1579.g663dfa89.el9 ceph 169 k 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:================================================================================ 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Install 1 Package 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 169 k 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Installed size: 540 k 2024-02-20T02:09:04.858 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-20T02:09:05.230 INFO:teuthology.orchestra.run.smithi125.stdout:rbd-nbd-19.0.0-1579.g663dfa89.el9.x86_64.rpm 455 kB/s | 169 kB 00:00 2024-02-20T02:09:05.231 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-20T02:09:05.231 INFO:teuthology.orchestra.run.smithi125.stdout:Total 453 kB/s | 169 kB 00:00 2024-02-20T02:09:05.231 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-20T02:09:05.239 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-20T02:09:05.239 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-20T02:09:05.300 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-20T02:09:05.300 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-20T02:09:05.454 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-20T02:09:05.565 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:06.026 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 1/1 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd-2:19.0.0-1579.g663dfa89.el9.x86_64 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:06.342 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-20T02:09:06.580 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:06.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:09:06.801 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-20T02:09:06.821 INFO:teuthology.orchestra.run.smithi125.stdout:19.0.0-1579.g663dfa89.el9 2024-02-20T02:09:06.822 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1579.g663dfa89.el9 2024-02-20T02:09:06.822 INFO:teuthology.task.install:The correct ceph version 19.0.0-1579.g663dfa89 is installed. 2024-02-20T02:09:06.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:09:06.965 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-20T02:09:06.990 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-1579.g663dfa89.el9 2024-02-20T02:09:06.990 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1579.g663dfa89.el9 2024-02-20T02:09:06.991 INFO:teuthology.task.install:The correct ceph version 19.0.0-1579.g663dfa89 is installed. 2024-02-20T02:09:06.992 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-20T02:09:06.993 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:06.993 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-20T02:09:07.021 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:07.021 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-20T02:09:07.062 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-20T02:09:07.062 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:07.062 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/daemon-helper 2024-02-20T02:09:07.088 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-20T02:09:07.155 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:07.155 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-02-20T02:09:07.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-20T02:09:07.255 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-20T02:09:07.255 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:07.255 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-20T02:09:07.281 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-20T02:09:07.348 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:07.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-20T02:09:07.376 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-20T02:09:07.446 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-20T02:09:07.446 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:07.446 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/stdin-killer 2024-02-20T02:09:07.473 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-20T02:09:07.540 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:07.540 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-02-20T02:09:07.568 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-20T02:09:07.635 INFO:teuthology.run_tasks:Running task ceph... 2024-02-20T02:09:07.717 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-02-20T02:09:07.717 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 777 /var/log/ceph 2024-02-20T02:09:07.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-02-20T02:09:07.746 INFO:tasks.ceph:Disabling ceph logrotate... 2024-02-20T02:09:07.746 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-20T02:09:07.787 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-20T02:09:07.813 INFO:tasks.ceph:Creating extra log directories... 2024-02-20T02:09:07.813 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-20T02:09:07.853 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-20T02:09:07.889 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-02-20T02:09:07.889 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-02-20T02:09:07.889 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238', 'branch': 'wip-sjust-crimson-testing-2024-02-19', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7568238', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-sjust-crimson-testing-2024-02-19'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c'}}, 'workunit': {'branch': 'wip-sjust-crimson-testing-2024-02-19', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c'}}, 'owner': 'scheduled_sjust@teuthology', 'priority': 51, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4769, 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-sjust-crimson-testing-2024-02-19', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c', 'targets': {'smithi125.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Qp7DETLHvlRbXOx2wpU4DdRiTNXdGoP1rxNFm4cBMOomqjvMx+/OclkALynycfzFVlOMbvPgxUrIL4pI3mi8=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Q8rcGTT41i9niKJxNu8yqnottToCTCJw8aBW7rEmy1tCZ0L0RzXTjJ4/8mOQVVvrgN1ksvraVhD+S5Pp337M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '663dfa8925d03992983e2ae809f7a5fc9835cd6c', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-02-20_01:50:29', 'tube': 'smithi', 'user': 'sjust', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-02-20T02:09:07.889 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-20T02:09:07.919 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-20T02:09:07.935 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m0777 -- /var/run/ceph 2024-02-20T02:09:07.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-02-20T02:09:08.002 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:08.002 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/scratch_devs of=/dev/stdout 2024-02-20T02:09:08.053 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-20T02:09:08.054 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_1 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-20 02:09:02.922116244 +0000 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-20 02:04:50.298639115 +0000 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-20 02:04:50.298639115 +0000 2024-02-20T02:09:08.108 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-20 02:04:50.298639115 +0000 2024-02-20T02:09:08.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-20T02:09:08.172 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-20T02:09:08.172 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-20T02:09:08.173 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000183313 s, 2.8 MB/s 2024-02-20T02:09:08.174 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-20T02:09:08.229 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_2 2024-02-20T02:09:08.284 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-20T02:09:08.284 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:08.284 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-20T02:09:08.284 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:08.285 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:08.285 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-20 02:09:02.922116244 +0000 2024-02-20T02:09:08.285 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-20 02:04:50.644633035 +0000 2024-02-20T02:09:08.285 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-20 02:04:50.644633035 +0000 2024-02-20T02:09:08.285 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-20 02:04:50.644633035 +0000 2024-02-20T02:09:08.285 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-20T02:09:08.349 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-20T02:09:08.350 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-20T02:09:08.350 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000190855 s, 2.7 MB/s 2024-02-20T02:09:08.351 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-20T02:09:08.406 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_3 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-20 02:09:02.922116244 +0000 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-20 02:04:50.988626990 +0000 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-20 02:04:50.988626990 +0000 2024-02-20T02:09:08.461 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-20 02:04:50.988626990 +0000 2024-02-20T02:09:08.462 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-20T02:09:08.525 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-20T02:09:08.525 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-20T02:09:08.525 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000212318 s, 2.4 MB/s 2024-02-20T02:09:08.526 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-20T02:09:08.581 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_4 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-20 02:09:02.923116227 +0000 2024-02-20T02:09:08.636 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-20 02:04:51.318621191 +0000 2024-02-20T02:09:08.637 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-20 02:04:51.318621191 +0000 2024-02-20T02:09:08.637 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-20 02:04:51.318621191 +0000 2024-02-20T02:09:08.637 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-20T02:09:08.702 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-20T02:09:08.702 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-20T02:09:08.702 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000196929 s, 2.6 MB/s 2024-02-20T02:09:08.703 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-20T02:09:08.758 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:08.758 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:08.758 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-02-20T02:09:08.774 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-20T02:09:08.774 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-02-20T02:09:08.828 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-20T02:09:08.828 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-02-20 02:09:00.237976889 +0000 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-02-20 02:04:50.298515023 +0000 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-02-20 02:04:50.298515023 +0000 2024-02-20T02:09:08.829 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-02-20 02:04:50.298515023 +0000 2024-02-20T02:09:08.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-20T02:09:08.894 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-02-20T02:09:08.895 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-02-20T02:09:08.895 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000185036 s, 2.8 MB/s 2024-02-20T02:09:08.896 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-20T02:09:08.951 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-02-20T02:09:09.006 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-20T02:09:09.006 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:09.006 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-20T02:09:09.006 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:09.006 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:09.007 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-02-20 02:09:00.237976889 +0000 2024-02-20T02:09:09.007 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-02-20 02:04:50.642510173 +0000 2024-02-20T02:09:09.007 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-02-20 02:04:50.642510173 +0000 2024-02-20T02:09:09.007 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-02-20 02:04:50.642510173 +0000 2024-02-20T02:09:09.007 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-20T02:09:09.072 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-02-20T02:09:09.072 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-02-20T02:09:09.072 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000216732 s, 2.4 MB/s 2024-02-20T02:09:09.073 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-20T02:09:09.128 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-02-20 02:09:00.238976875 +0000 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-02-20 02:04:50.975505479 +0000 2024-02-20T02:09:09.183 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-02-20 02:04:50.975505479 +0000 2024-02-20T02:09:09.184 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-02-20 02:04:50.975505479 +0000 2024-02-20T02:09:09.184 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-20T02:09:09.249 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-02-20T02:09:09.249 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-02-20T02:09:09.249 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000207733 s, 2.5 MB/s 2024-02-20T02:09:09.250 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-20T02:09:09.306 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-02-20 02:09:00.238976875 +0000 2024-02-20T02:09:09.361 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-02-20 02:04:51.331500460 +0000 2024-02-20T02:09:09.362 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-02-20 02:04:51.331500460 +0000 2024-02-20T02:09:09.362 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-02-20 02:04:51.331500460 +0000 2024-02-20T02:09:09.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-20T02:09:09.426 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-02-20T02:09:09.427 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-02-20T02:09:09.427 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000203658 s, 2.5 MB/s 2024-02-20T02:09:09.428 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-20T02:09:09.483 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:09.483 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-02-20T02:09:09.483 INFO:tasks.ceph:Generating config... 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] debug mon = 20 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] debug ms = 1 2024-02-20T02:09:09.484 INFO:tasks.ceph:[mon] debug paxos = 20 2024-02-20T02:09:09.485 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-02-20T02:09:09.485 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] debug monc = 20 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] debug ms = 1 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] debug osd = 20 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] osd class default list = * 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] osd class load list = * 2024-02-20T02:09:09.485 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-02-20T02:09:09.485 INFO:tasks.ceph:[client] debug ms = 1 2024-02-20T02:09:09.485 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-02-20T02:09:09.485 INFO:tasks.ceph:[global] ms client mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[global] ms service mode = crc 2024-02-20T02:09:09.486 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-02-20T02:09:09.486 INFO:tasks.ceph:[mgr] debug ms = 1 2024-02-20T02:09:09.486 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.486 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.486 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.486 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.486 INFO:tasks.ceph:Setting up mon.a... 2024-02-20T02:09:09.487 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-02-20T02:09:09.548 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.keyring 2024-02-20T02:09:09.551 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-02-20T02:09:09.601 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-20T02:09:09.627 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.125')] 2024-02-20T02:09:09.627 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.125', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-02-20T02:09:09.628 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:09.628 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-02-20T02:09:09.681 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.125 --print /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:09.766 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:09.766+0000 7f1edfa31e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:09.766 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:09.766+0000 7f1edfa31e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:09.766 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:09.766+0000 7f1edfa31e80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: generated fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:setting min_mon_release = quincy 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:epoch 0 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:last_changed 2024-02-20T02:09:09.768990+0000 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:created 2024-02-20T02:09:09.768990+0000 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:min_mon_release 17 (quincy) 2024-02-20T02:09:09.767 INFO:teuthology.orchestra.run.smithi125.stdout:election_strategy: 1 2024-02-20T02:09:09.768 INFO:teuthology.orchestra.run.smithi125.stdout:0: [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] mon.a 2024-02-20T02:09:09.768 INFO:teuthology.orchestra.run.smithi125.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-02-20T02:09:09.769 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-02-20T02:09:09.783 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7c271077-0df8-4ad5-a525-6f111846375d... 2024-02-20T02:09:09.784 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-20T02:09:09.825 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout:[global] 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: chdir = "" 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: auth supported = cephx 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: filestore xattr use omap = true 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: mon clock drift allowed = 1.000 2024-02-20T02:09:09.873 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: osd crush chooseleaf type = 0 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: auth debug = true 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on old message = true 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: ms die on bug = true 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: mon pg warn max object skew = 0 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: # disable pg_autoscaler by default for new pools 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default size = 2 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow primary affinity = true 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd allow pg remap = true 2024-02-20T02:09:09.874 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on legacy crush tunables = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on crush straw calc version zero = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on no sortbitwise = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on osd down out interval zero = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on too few osds = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon_allow_pool_size_one = true 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: osd default data pool replay window = 5 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: mon allow pool delete = true 2024-02-20T02:09:09.875 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: mon cluster log file level = debug 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: debug asserts on shutdown = true 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: mon health detail to clog = false 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: mon host = 172.21.15.125 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms client mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms cluster mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon client mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon cluster mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms mon service mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: ms service mode = crc 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: fsid = 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout:[osd] 2024-02-20T02:09:09.876 INFO:teuthology.orchestra.run.smithi125.stdout: osd journal size = 100 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub load threshold = 5.0 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd scrub max interval = 600 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd mclock profile = high_recovery_ops 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd recover clone overlap = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd recovery max chunk = 1048576 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug shutdown = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug op order = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug verify stray on activate = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug trim objects = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd open classes on start = true 2024-02-20T02:09:09.877 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug pg log writeout = true 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: osd deep scrub update digest min age = 30 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: osd map max advance = 10 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: journal zero on create = true 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: filestore ondisk finisher threads = 3 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: filestore apply finisher threads = 3 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: bdev debug aio = true 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: osd debug misdirected ops = true 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: debug monc = 20 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: crimson alien thread cpu cores = 6-7 2024-02-20T02:09:09.878 INFO:teuthology.orchestra.run.smithi125.stdout: crimson osd obc lru size = 10 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug osd = 20 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: osd blocked scrub grace period = 3600 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: osd class default list = * 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: osd class load list = * 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: osd shutdown pgref assert = True 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr] 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug mgr = 20 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2024-02-20T02:09:09.879 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mgr/telemetry/nag = false 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout:[mon] 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: debug mon = 20 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: debug paxos = 20 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: debug auth = 20 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon data avail warn = 5 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon mgr mkfs grace = 240 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min pgs per osd = 4 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd reporter subtree level = osd 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon osd prime pg temp = true 2024-02-20T02:09:09.880 INFO:teuthology.orchestra.run.smithi125.stdout: mon reweight min bytes per osd = 10 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: auth mon ticket ttl = 660 # 11m 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: auth service ticket ttl = 240 # 4m 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: # don't complain about insecure global_id in the test suite 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: # 1m isn't quite enough 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: mon_down_mkfs_grace = 2m 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: mon_warn_on_filestore_osds = false 2024-02-20T02:09:09.881 INFO:teuthology.orchestra.run.smithi125.stdout: mon min osdmap epochs = 50 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune interval = 2 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune min = 15 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: mon osdmap full prune txsize = 2 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: paxos service trim min = 10 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: mon warn on pool no app = False 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: osd pool default crimson = True 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout:[client] 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: rgw cache enabled = true 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable ops log = true 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: rgw enable usage log = true 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-20T02:09:09.882 INFO:teuthology.orchestra.run.smithi125.stdout: debug ms = 1 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout:[mon.a] 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.0] 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout: crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.1] 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout: crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.2] 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout: crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.3] 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi125.stdout: crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.883 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-02-20T02:09:09.884 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-02-20T02:09:09.885 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = 172.21.15.125 2024-02-20T02:09:09.886 INFO:teuthology.orchestra.run.smithi204.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms client mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms cluster mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms mon client mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms mon cluster mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms mon service mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: ms service mode = crc 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.887 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-02-20T02:09:09.888 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: debug monc = 20 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: crimson alien thread cpu cores = 6-7 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: crimson osd obc lru size = 10 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: osd blocked scrub grace period = 3600 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: osd class default list = * 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: osd class load list = * 2024-02-20T02:09:09.889 INFO:teuthology.orchestra.run.smithi204.stdout: osd shutdown pgref assert = True 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-02-20T02:09:09.890 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-20T02:09:09.891 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon min osdmap epochs = 50 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune interval = 2 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune min = 15 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune txsize = 2 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: paxos service trim min = 10 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on pool no app = False 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default crimson = True 2024-02-20T02:09:09.892 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout: crimson seastar cpu cores = 0-2 2024-02-20T02:09:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-02-20T02:09:09.894 INFO:teuthology.orchestra.run.smithi204.stdout: crimson seastar cpu cores = 3-5 2024-02-20T02:09:09.895 INFO:tasks.ceph:Creating admin key on mon.a... 2024-02-20T02:09:09.895 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-02-20T02:09:09.956 INFO:tasks.ceph:Copying monmap to all nodes... 2024-02-20T02:09:09.956 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:09.956 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-02-20T02:09:09.970 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:09.970 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-02-20T02:09:10.025 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:09:10.025 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:10.025 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-20T02:09:10.025 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-20T02:09:10.104 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:10.105 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:10.159 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:09:10.159 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:10.160 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-20T02:09:10.160 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-20T02:09:10.198 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:10.199 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:10.253 INFO:tasks.ceph:Setting up mon nodes... 2024-02-20T02:09:10.253 INFO:tasks.ceph:Setting up mgr nodes... 2024-02-20T02:09:10.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-02-20T02:09:10.364 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-02-20T02:09:10.368 INFO:tasks.ceph:Setting up mds nodes... 2024-02-20T02:09:10.368 INFO:tasks.ceph_client:Setting up client nodes... 2024-02-20T02:09:10.368 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-20T02:09:10.417 INFO:teuthology.orchestra.run.smithi125.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-02-20T02:09:10.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-20T02:09:10.481 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-02-20T02:09:10.497 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-02-20T02:09:10.497 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi125.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-02-20T02:09:10.497 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-02-20T02:09:10.523 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:10.523 INFO:tasks.ceph:role: osd.0 2024-02-20T02:09:10.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:09:10.524 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-20T02:09:10.597 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-20T02:09:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-20T02:09:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-20T02:09:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-20T02:09:10.597 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-20T02:09:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2024-02-20T02:09:10.598 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-20T02:09:10.598 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-20T02:09:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-20T02:09:10.598 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-20T02:09:11.106 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2024-02-20T02:09:11.108 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2024-02-20T02:09:11.108 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-02-20T02:09:11.238 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-02-20T02:09:11.271 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-02-20T02:09:11.338 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:11.338 INFO:tasks.ceph:role: osd.1 2024-02-20T02:09:11.338 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:09:11.338 DEBUG:teuthology.orchestra.run.smithi125:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout: = sunit=0 swidth=0 blks 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-20T02:09:11.413 INFO:teuthology.orchestra.run.smithi125.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-20T02:09:11.414 INFO:teuthology.orchestra.run.smithi125.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-20T02:09:11.894 INFO:teuthology.orchestra.run.smithi125.stdout:Discarding blocks...Done. 2024-02-20T02:09:11.896 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi125.front.sepia.ceph.com -o noatime 2024-02-20T02:09:11.896 DEBUG:teuthology.orchestra.run.smithi125:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-02-20T02:09:11.998 DEBUG:teuthology.orchestra.run.smithi125:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-02-20T02:09:12.029 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:12.579 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:12.580 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:12,582 seastar - Reactor backend: linux-aio 2024-02-20T02:09:12.602 INFO:teuthology.orchestra.run.smithi125.stderr:==28669==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:12.608 INFO:teuthology.orchestra.run.smithi125.stderr:==28669==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcebcf1000; bottom 0x7f24ac8ba000; size: 0x00d83f437000 (928774320128) 2024-02-20T02:09:12.608 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2024-02-20T02:09:12.608 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:12,650 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-02-20T02:09:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:12,650 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-02-20T02:09:12.648 INFO:teuthology.orchestra.run.smithi125.stderr:DEBUG 2024-02-20 02:09:12,650 [shard 0:main] seastar - reactor::drain 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:13.123 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-20T02:09:13.176 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:13.177 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:13,180 seastar - Reactor backend: linux-aio 2024-02-20T02:09:13.205 INFO:teuthology.orchestra.run.smithi125.stderr:==28667==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:13.211 INFO:teuthology.orchestra.run.smithi125.stderr:==28667==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcebcf1000; bottom 0x7f24aa6a5000; size: 0x00d84164c000 (928810057728) 2024-02-20T02:09:13.211 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2024-02-20T02:09:13.211 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:16.245 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-20T02:09:16.246 INFO:teuthology.orchestra.run.smithi125.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:16.285 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-20T02:09:16.312 DEBUG:teuthology.orchestra.run.smithi125:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:16.838 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:16,840 seastar - Reactor backend: linux-aio 2024-02-20T02:09:16.860 INFO:teuthology.orchestra.run.smithi125.stderr:==29478==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:==29478==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc03069000; bottom 0x7f0fef686000; size: 0x00ec139e3000 (1013941415936) 2024-02-20T02:09:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2024-02-20T02:09:16.866 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:16.902 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:16,904 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-02-20T02:09:16.902 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:16,904 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-02-20T02:09:16.903 INFO:teuthology.orchestra.run.smithi125.stderr:DEBUG 2024-02-20 02:09:16,905 [shard 0:main] seastar - reactor::drain 2024-02-20T02:09:17.373 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:17.373 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2024-02-20T02:09:17.374 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2024-02-20T02:09:17.374 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:17.374 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:17.374 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-20T02:09:17.429 INFO:teuthology.orchestra.run.smithi125.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:17.430 INFO:teuthology.orchestra.run.smithi125.stderr:INFO 2024-02-20 02:09:17,433 seastar - Reactor backend: linux-aio 2024-02-20T02:09:17.459 INFO:teuthology.orchestra.run.smithi125.stderr:==29476==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:17.464 INFO:teuthology.orchestra.run.smithi125.stderr:==29476==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc03069000; bottom 0x7f0fed471000; size: 0x00ec15bf8000 (1013977153536) 2024-02-20T02:09:17.464 INFO:teuthology.orchestra.run.smithi125.stderr:False positive error reports may follow 2024-02-20T02:09:17.464 INFO:teuthology.orchestra.run.smithi125.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr:Suppressions used: 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr: count bytes template 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr:----------------------------------------------------- 2024-02-20T02:09:20.553 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-20T02:09:20.554 INFO:teuthology.orchestra.run.smithi125.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:20.595 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-20T02:09:20.621 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-02-20T02:09:20.653 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:20.654 INFO:tasks.ceph:role: osd.2 2024-02-20T02:09:20.654 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:09:20.654 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-20T02:09:20.731 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-20T02:09:20.731 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-20T02:09:20.731 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-20T02:09:20.731 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-20T02:09:20.731 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-20T02:09:20.732 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-02-20T02:09:20.732 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-20T02:09:20.732 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-20T02:09:20.732 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-20T02:09:20.732 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-20T02:09:21.262 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-02-20T02:09:21.264 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-02-20T02:09:21.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-02-20T02:09:21.377 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-02-20T02:09:21.413 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-02-20T02:09:21.482 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-02-20T02:09:21.482 INFO:tasks.ceph:role: osd.3 2024-02-20T02:09:21.482 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:09:21.482 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-20T02:09:21.558 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-20T02:09:21.559 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-20T02:09:21.998 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-02-20T02:09:22.000 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-02-20T02:09:22.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-02-20T02:09:22.094 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-02-20T02:09:22.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:22.683 INFO:teuthology.orchestra.run.smithi204.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:22.684 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:22,686 seastar - Reactor backend: linux-aio 2024-02-20T02:09:22.703 INFO:teuthology.orchestra.run.smithi204.stderr:==28427==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:22.709 INFO:teuthology.orchestra.run.smithi204.stderr:==28427==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff3f836000; bottom 0x7f4c4fe3b000; size: 0x00b2ef9fb000 (768524398592) 2024-02-20T02:09:22.710 INFO:teuthology.orchestra.run.smithi204.stderr:False positive error reports may follow 2024-02-20T02:09:22.710 INFO:teuthology.orchestra.run.smithi204.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:22.749 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:22,751 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-02-20T02:09:22.749 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:22,751 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-02-20T02:09:22.750 INFO:teuthology.orchestra.run.smithi204.stderr:DEBUG 2024-02-20 02:09:22,752 [shard 0:main] seastar - reactor::drain 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr:Suppressions used: 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr: count bytes template 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:23.239 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-02-20T02:09:23.299 INFO:teuthology.orchestra.run.smithi204.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:23.300 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:23,303 seastar - Reactor backend: linux-aio 2024-02-20T02:09:23.328 INFO:teuthology.orchestra.run.smithi204.stderr:==28423==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:23.334 INFO:teuthology.orchestra.run.smithi204.stderr:==28423==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff3f836000; bottom 0x7f4c4dc26000; size: 0x00b2f1c10000 (768560136192) 2024-02-20T02:09:23.334 INFO:teuthology.orchestra.run.smithi204.stderr:False positive error reports may follow 2024-02-20T02:09:23.334 INFO:teuthology.orchestra.run.smithi204.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr:Suppressions used: 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr: count bytes template 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-02-20T02:09:26.418 INFO:teuthology.orchestra.run.smithi204.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:26.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-20T02:09:26.487 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:27.011 INFO:teuthology.orchestra.run.smithi204.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:27.011 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:27,014 seastar - Reactor backend: linux-aio 2024-02-20T02:09:27.029 INFO:teuthology.orchestra.run.smithi204.stderr:==29236==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:27.035 INFO:teuthology.orchestra.run.smithi204.stderr:==29236==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff9ce17000; bottom 0x7f901113f000; size: 0x006f8bcd8000 (479086870528) 2024-02-20T02:09:27.036 INFO:teuthology.orchestra.run.smithi204.stderr:False positive error reports may follow 2024-02-20T02:09:27.036 INFO:teuthology.orchestra.run.smithi204.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:27.073 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:27,075 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-02-20T02:09:27.073 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:27,075 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-02-20T02:09:27.073 INFO:teuthology.orchestra.run.smithi204.stderr:DEBUG 2024-02-20 02:09:27,076 [shard 0:main] seastar - reactor::drain 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr:Suppressions used: 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr: count bytes template 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:27.542 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-02-20T02:09:27.598 INFO:teuthology.orchestra.run.smithi204.stderr:WARNING: debug mode. Not for benchmarking or production 2024-02-20T02:09:27.599 INFO:teuthology.orchestra.run.smithi204.stderr:INFO 2024-02-20 02:09:27,602 seastar - Reactor backend: linux-aio 2024-02-20T02:09:27.628 INFO:teuthology.orchestra.run.smithi204.stderr:==29234==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-02-20T02:09:27.633 INFO:teuthology.orchestra.run.smithi204.stderr:==29234==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff9ce17000; bottom 0x7f900ef2a000; size: 0x006f8deed000 (479122608128) 2024-02-20T02:09:27.633 INFO:teuthology.orchestra.run.smithi204.stderr:False positive error reports may follow 2024-02-20T02:09:27.633 INFO:teuthology.orchestra.run.smithi204.stderr:For details see https://github.com/google/sanitizers/issues/189 2024-02-20T02:09:30.728 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:30.728 INFO:teuthology.orchestra.run.smithi204.stderr:Suppressions used: 2024-02-20T02:09:30.728 INFO:teuthology.orchestra.run.smithi204.stderr: count bytes template 2024-02-20T02:09:30.729 INFO:teuthology.orchestra.run.smithi204.stderr: 1 8 MallocExtension::Register 2024-02-20T02:09:30.729 INFO:teuthology.orchestra.run.smithi204.stderr:----------------------------------------------------- 2024-02-20T02:09:30.729 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-02-20T02:09:30.729 INFO:teuthology.orchestra.run.smithi204.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid 7c271077-0df8-4ad5-a525-6f111846375d 2024-02-20T02:09:30.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-20T02:09:30.797 INFO:tasks.ceph:Reading keys from all nodes... 2024-02-20T02:09:30.797 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:30.797 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-02-20T02:09:30.829 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:30.829 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-02-20T02:09:30.898 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:30.899 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-02-20T02:09:30.925 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:30.926 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-02-20T02:09:30.992 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:30.992 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-02-20T02:09:31.059 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:31.059 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-02-20T02:09:31.076 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:31.076 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-02-20T02:09:31.114 INFO:tasks.ceph:Adding keys to all mons... 2024-02-20T02:09:31.114 DEBUG:teuthology.orchestra.run.smithi125:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.0] 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDJCdRl5UquDxAApo4OofUrxJvwbWPJp3YaxQ== 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.1] 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDNCdRlQlLJHhAA9NAcwtVq4EcqnGUnZo0ypQ== 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout:[mgr.x] 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDGCdRl107iFRAASTrOkMqR/+RCpXx17ljgkg== 2024-02-20T02:09:31.145 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.2] 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDTCdRlfo8YFxAA6oZPXs/mhvrz5bBx61+fAQ== 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout:[osd.3] 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDXCdRl8affKBAAaHzPj7BuIWa0UcnTxFsbbA== 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout:[client.0] 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDGCdRl23cFGRAAzmXBnuPqD2V1+5Pzyd5OIA== 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout:[client.1] 2024-02-20T02:09:31.146 INFO:teuthology.orchestra.run.smithi125.stdout: key = AQDGCdRluPzXHBAAlYIsFvH7trOT09Wt2mixDA== 2024-02-20T02:09:31.147 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-20T02:09:31.248 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-20T02:09:31.306 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-20T02:09:31.364 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-20T02:09:31.422 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-20T02:09:31.481 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-20T02:09:31.543 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-20T02:09:31.605 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-02-20T02:09:31.605 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-02-20T02:09:31.631 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-20T02:09:31.732 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.733+0000 7f39333119c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.732 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.733+0000 7f39333119c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.747 DEBUG:teuthology.orchestra.run.smithi125:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-02-20T02:09:31.774 DEBUG:teuthology.orchestra.run.smithi125:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-20T02:09:31.828 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-02-20T02:09:31.828 INFO:tasks.ceph.mon.a:Restarting daemon 2024-02-20T02:09:31.829 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-02-20T02:09:31.870 INFO:tasks.ceph.mon.a:Started 2024-02-20T02:09:31.870 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-02-20T02:09:31.871 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-02-20T02:09:31.871 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-02-20T02:09:31.873 INFO:tasks.ceph.mgr.x:Started 2024-02-20T02:09:31.873 INFO:tasks.ceph:Setting crush tunables to default 2024-02-20T02:09:31.873 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd crush tunables default 2024-02-20T02:09:31.949 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:09:31.950+0000 7f14c2a659c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.949 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:09:31.950+0000 7f14c2a659c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.949 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:09:31.950+0000 7f14c2a659c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.965 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:31.966+0000 7fedbf00bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.978+0000 7fa94606d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:31.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.984+0000 7fa94606d640 1 Processor -- start 2024-02-20T02:09:31.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.984+0000 7fa94606d640 1 -- start start 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa94606d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa94606d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94009a810 con 0x7fa9400a17d0 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa94606d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa940099f20 con 0x7fa9401342a0 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93effd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fa9400a17d0 0x7fa9400a1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36210/0 (socket says 172.21.15.125:36210) 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37138/0 (socket says 172.21.15.125:37138) 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93effd640 1 -- 172.21.15.125:0/2664255901 learned_addr learned my addr 172.21.15.125:0/2664255901 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 -- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 msgr2=0x7fa9401346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 -- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 msgr2=0x7fa9401346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 --2- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-20T02:09:31.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:31.985+0000 7fa93f7fe640 1 --2- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-20T02:09:32.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.012+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117019062 0 0) 0x7fa94009a810 con 0x7fa9400a17d0 2024-02-20T02:09:32.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.012+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9280034f0 con 0x7fa9400a17d0 2024-02-20T02:09:32.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v1:172.21.15.125:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802019046 0 0) 0x7fa9280034f0 con 0x7fa9400a17d0 2024-02-20T02:09:32.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 msgr2=0x7fa9401346b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-20T02:09:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 --2- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 --> v1:172.21.15.125:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94009bbe0 con 0x7fa9400a17d0 2024-02-20T02:09:32.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v1:172.21.15.125:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 3980158998 0 0) 0x7fa9300015a0 con 0x7fa9400a17d0 2024-02-20T02:09:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 >> v1:172.21.15.125:6789/0 conn(0x7fa9400a17d0 legacy=0x7fa9400a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 --2- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa928003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa940099f20 con 0x7fa928003940 2024-02-20T02:09:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93f7fe640 1 --2- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa928003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:32.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.013+0000 7fa93f7fe640 1 -- 172.21.15.125:0/2664255901 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280034f0 con 0x7fa928003940 2024-02-20T02:09:32.015 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.015+0000 7fa93f7fe640 1 --2- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa928003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd056bd789c3d595 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.015+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa93400b5b0 con 0x7fa928003940 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.015+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa93400baf0 con 0x7fa928003940 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.015+0000 7fa93e7fc640 1 -- 172.21.15.125:0/2664255901 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa93400bd90 con 0x7fa928003940 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 -- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 msgr2=0x7fa928003d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 --2- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa928003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 -- 172.21.15.125:0/2664255901 shutdown_connections 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 --2- 172.21.15.125:0/2664255901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa928003d90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 --2- 172.21.15.125:0/2664255901 >> v2:172.21.15.125:3300/0 conn(0x7fa9401342a0 0x7fa9401346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 -- 172.21.15.125:0/2664255901 >> 172.21.15.125:0/2664255901 conn(0x7fa94006c8a0 msgr2=0x7fa94006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 -- 172.21.15.125:0/2664255901 shutdown_connections 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 1 -- 172.21.15.125:0/2664255901 wait complete. 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.016+0000 7fa94606d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.017+0000 7fa94606d640 1 Processor -- start 2024-02-20T02:09:32.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.017+0000 7fa94606d640 1 -- start start 2024-02-20T02:09:32.017 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:32.016+0000 7fedbf00bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:32.017 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:32.016+0000 7fedbf00bf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa94606d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa94606d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94011e2b0 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa93f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa93f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37152/0 (socket says 172.21.15.125:37152) 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa93f7fe640 1 -- 172.21.15.125:0/1451910456 learned_addr learned my addr 172.21.15.125:0/1451910456 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa93f7fe640 1 -- 172.21.15.125:0/1451910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9401c5200 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa93f7fe640 1 --2- 172.21.15.125:0/1451910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.018+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa9340035e0 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa934003740 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa9340039e0 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94011f630 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94011fad0 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7fa934016430 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.019+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (crc 0 0 0) 0x7fa934015b70 con 0x7fa928003940 2024-02-20T02:09:32.019 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.020+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940062a30 con 0x7fa928003940 2024-02-20T02:09:32.021 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.022+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7fa934015dd0 con 0x7fa928003940 2024-02-20T02:09:32.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.112+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa9401c5390 con 0x7fa928003940 2024-02-20T02:09:32.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.113+0000 7fa927fff640 1 -- 172.21.15.125:0/1451910456 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (crc 0 0 0) 0x7fa9340168c0 con 0x7fa928003940 2024-02-20T02:09:32.111 INFO:teuthology.orchestra.run.smithi125.stderr:adjusted tunables profile to default 2024-02-20T02:09:32.112 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.113+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 msgr2=0x7fa9401c55a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.113+0000 7fa94606d640 1 --2- 172.21.15.125:0/1451910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.113+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 shutdown_connections 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.113+0000 7fa94606d640 1 --2- 172.21.15.125:0/1451910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa928003940 0x7fa9401c55a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.114+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 >> 172.21.15.125:0/1451910456 conn(0x7fa94006c8a0 msgr2=0x7fa9401bbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.114+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 shutdown_connections 2024-02-20T02:09:32.113 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.114+0000 7fa94606d640 1 -- 172.21.15.125:0/1451910456 wait complete. 2024-02-20T02:09:32.122 INFO:tasks.ceph:check_enable_crimson: True 2024-02-20T02:09:32.123 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-02-20T02:09:32.123 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-02-20T02:09:32.227 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.224+0000 7f651539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:32.228 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.229+0000 7f651539d640 1 Processor -- start 2024-02-20T02:09:32.228 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.229+0000 7f651539d640 1 -- start start 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f651539d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f651012d4a0 0x7f6510135270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f651539d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651009b710 con 0x7f651009bea0 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f651539d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651006eb50 con 0x7f651012d4a0 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f6507fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f651012d4a0 0x7f6510135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f650effd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f651009bea0 0x7f651012cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36224/0 (socket says 172.21.15.125:36224) 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f650effd640 1 -- 172.21.15.125:0/659904352 learned_addr learned my addr 172.21.15.125:0/659904352 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f6507fff640 1 -- 172.21.15.125:0/659904352 >> v1:172.21.15.125:6789/0 conn(0x7f651009bea0 legacy=0x7f651012cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:32.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f6507fff640 1 -- 172.21.15.125:0/659904352 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651006e070 con 0x7f651012d4a0 2024-02-20T02:09:32.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f6507fff640 1 --2- 172.21.15.125:0/659904352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651012d4a0 0x7f6510135270 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2933f2bf7e3cd76b server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:32.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f650e7fc640 1 -- 172.21.15.125:0/659904352 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64f8009ce0 con 0x7f651012d4a0 2024-02-20T02:09:32.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f650e7fc640 1 -- 172.21.15.125:0/659904352 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f64f8009e40 con 0x7f651012d4a0 2024-02-20T02:09:32.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.230+0000 7f650e7fc640 1 -- 172.21.15.125:0/659904352 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64f800e360 con 0x7f651012d4a0 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 -- 172.21.15.125:0/659904352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651012d4a0 msgr2=0x7f6510135270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 --2- 172.21.15.125:0/659904352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651012d4a0 0x7f6510135270 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 -- 172.21.15.125:0/659904352 shutdown_connections 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 --2- 172.21.15.125:0/659904352 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651012d4a0 0x7f6510135270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 -- 172.21.15.125:0/659904352 >> 172.21.15.125:0/659904352 conn(0x7f651009fae0 msgr2=0x7f651009fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 -- 172.21.15.125:0/659904352 shutdown_connections 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 1 -- 172.21.15.125:0/659904352 wait complete. 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.231+0000 7f651539d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.232+0000 7f651539d640 1 Processor -- start 2024-02-20T02:09:32.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.232+0000 7f651539d640 1 -- start start 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f651539d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f651539d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101c68b0 con 0x7f651009bea0 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f650effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f650effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37160/0 (socket says 172.21.15.125:37160) 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f650effd640 1 -- 172.21.15.125:0/4246074868 learned_addr learned my addr 172.21.15.125:0/4246074868 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f650effd640 1 -- 172.21.15.125:0/4246074868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651011ee40 con 0x7f651009bea0 2024-02-20T02:09:32.233 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f650effd640 1 --2- 172.21.15.125:0/4246074868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.233+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f65000059d0 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6500005b30 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6500005dd0 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651009d760 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651009d3e0 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f6500007440 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (crc 0 0 0) 0x7f650000f7f0 con 0x7f651009bea0 2024-02-20T02:09:32.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.234+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510062a30 con 0x7f651009bea0 2024-02-20T02:09:32.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.237+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f650000fa50 con 0x7f651009bea0 2024-02-20T02:09:32.311 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:32.312+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f651011de00 con 0x7f651009bea0 2024-02-20T02:09:32.311 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:09:32.313+0000 7f14ba9f1640 -1 WARNING: experimental feature 'crimson' is enabled 2024-02-20T02:09:32.311 INFO:tasks.ceph.mon.a.smithi125.stderr:Please be aware that this feature is experimental, untested, 2024-02-20T02:09:32.312 INFO:tasks.ceph.mon.a.smithi125.stderr:unsupported, and may result in data corruption, data loss, 2024-02-20T02:09:32.312 INFO:tasks.ceph.mon.a.smithi125.stderr:and/or irreparable damage to your cluster. Do not use 2024-02-20T02:09:32.312 INFO:tasks.ceph.mon.a.smithi125.stderr:feature with important data. 2024-02-20T02:09:32.312 INFO:tasks.ceph.mon.a.smithi125.stderr: 2024-02-20T02:09:32.502 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:32.504+0000 7fedbf00bf00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-20T02:09:32.557 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:32.558+0000 7fedbf00bf00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-20T02:09:33.112 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.114+0000 7f65077fe640 1 -- 172.21.15.125:0/4246074868 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) v1 ==== 99+0+0 (crc 0 0 0) 0x7f650000fcc0 con 0x7f651009bea0 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 msgr2=0x7f65101c6c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 --2- 172.21.15.125:0/4246074868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 shutdown_connections 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 --2- 172.21.15.125:0/4246074868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f651009bea0 0x7f65101c6c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 >> 172.21.15.125:0/4246074868 conn(0x7f651009fae0 msgr2=0x7f651012fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 shutdown_connections 2024-02-20T02:09:33.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:33.115+0000 7f651539d640 1 -- 172.21.15.125:0/4246074868 wait complete. 2024-02-20T02:09:33.120 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.121+0000 7fedbf00bf00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-20T02:09:33.126 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-02-20T02:09:33.126 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:33.126 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-02-20T02:09:33.155 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:09:33.155 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-02-20T02:09:33.174 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.176+0000 7fedbf00bf00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-20T02:09:33.220 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.221+0000 7fedbf00bf00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-20T02:09:33.224 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:33.224 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-02-20T02:09:33.251 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-02-20T02:09:33.251 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-02-20T02:09:33.277 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.278+0000 7fedbf00bf00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-20T02:09:33.304 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 38094573-2a02-424b-9ba5-a0e77fed1db3 0 2024-02-20T02:09:33.323 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.324+0000 7fedbf00bf00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-20T02:09:33.429 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.425+0000 7f1530d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.431+0000 7f1530d9d640 1 Processor -- start 2024-02-20T02:09:33.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.431+0000 7f1530d9d640 1 -- start start 2024-02-20T02:09:33.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.431+0000 7f1530d9d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f152c11b8e0 0x7f152c11dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.431+0000 7f1530d9d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c11e2d0 con 0x7f152c112c90 2024-02-20T02:09:33.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.431+0000 7f1530d9d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c11f470 con 0x7f152c11b8e0 2024-02-20T02:09:33.431 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1523fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f152c11b8e0 0x7f152c11dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f152a59c640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f152c112c90 0x7f152c1130a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.204:42108/0 (socket says 172.21.15.204:42108) 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f152a59c640 1 -- 172.21.15.204:0/383679710 learned_addr learned my addr 172.21.15.204:0/383679710 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1523fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f152c11b8e0 0x7f152c11dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33768/0 (socket says 172.21.15.204:33768) 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1529d9b640 1 -- 172.21.15.204:0/383679710 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630406340 0 0) 0x7f152c11e2d0 con 0x7f152c112c90 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1529d9b640 1 -- 172.21.15.204:0/383679710 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15180034f0 con 0x7f152c112c90 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1523fff640 1 -- 172.21.15.204:0/383679710 >> v1:172.21.15.125:6789/0 conn(0x7f152c112c90 legacy=0x7f152c1130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1523fff640 1 -- 172.21.15.204:0/383679710 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c11f5b0 con 0x7f152c11b8e0 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1523fff640 1 --2- 172.21.15.204:0/383679710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c11b8e0 0x7f152c11dd00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ae3bd4c4fdab8c server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1529d9b640 1 -- 172.21.15.204:0/383679710 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1514009ce0 con 0x7f152c11b8e0 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1529d9b640 1 -- 172.21.15.204:0/383679710 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1514009e40 con 0x7f152c11b8e0 2024-02-20T02:09:33.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.432+0000 7f1529d9b640 1 -- 172.21.15.204:0/383679710 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f151400e360 con 0x7f152c11b8e0 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 -- 172.21.15.204:0/383679710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c11b8e0 msgr2=0x7f152c11dd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 --2- 172.21.15.204:0/383679710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c11b8e0 0x7f152c11dd00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 -- 172.21.15.204:0/383679710 shutdown_connections 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 --2- 172.21.15.204:0/383679710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c11b8e0 0x7f152c11dd00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 -- 172.21.15.204:0/383679710 >> 172.21.15.204:0/383679710 conn(0x7f152c06d8a0 msgr2=0x7f152c110dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 -- 172.21.15.204:0/383679710 shutdown_connections 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 1 -- 172.21.15.204:0/383679710 wait complete. 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.433+0000 7f1530d9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.434+0000 7f1530d9d640 1 Processor -- start 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.434+0000 7f1530d9d640 1 -- start start 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.434+0000 7f1530d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.434+0000 7f1530d9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1ad950 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f152a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f152a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33784/0 (socket says 172.21.15.204:33784) 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f152a59c640 1 -- 172.21.15.204:0/2873749073 learned_addr learned my addr 172.21.15.204:0/2873749073 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f152a59c640 1 -- 172.21.15.204:0/2873749073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c1ada90 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f152a59c640 1 --2- 172.21.15.204:0/2873749073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f151c0059d0 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f151c005b30 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c1adce0 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f151c005dd0 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c1a74d0 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f151c007440 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.435+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 747+0+0 (crc 0 0 0) 0x7f151c00f9e0 con 0x7f152c112c90 2024-02-20T02:09:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.436+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c1aded0 con 0x7f152c112c90 2024-02-20T02:09:33.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.439+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f151c00f230 con 0x7f152c112c90 2024-02-20T02:09:33.447 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.448+0000 7fedbf00bf00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-20T02:09:33.510 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.511+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "38094573-2a02-424b-9ba5-a0e77fed1db3", "id": 0} v 0) v1 -- 0x7f152c066ef0 con 0x7f152c112c90 2024-02-20T02:09:33.511 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.513+0000 7f1522ffd640 1 -- 172.21.15.204:0/2873749073 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38094573-2a02-424b-9ba5-a0e77fed1db3", "id": 0}]=0 v4) v1 ==== 112+0+2 (crc 0 0 0) 0x7f151c00fd10 con 0x7f152c112c90 2024-02-20T02:09:33.512 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-02-20T02:09:33.512 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 msgr2=0x7f152c1ad380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 --2- 172.21.15.204:0/2873749073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 shutdown_connections 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 --2- 172.21.15.204:0/2873749073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f152c112c90 0x7f152c1ad380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 >> 172.21.15.204:0/2873749073 conn(0x7f152c06d8a0 msgr2=0x7f152c11bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 shutdown_connections 2024-02-20T02:09:33.513 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.514+0000 7f1530d9d640 1 -- 172.21.15.204:0/2873749073 wait complete. 2024-02-20T02:09:33.520 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.522+0000 7fedbf00bf00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-20T02:09:33.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 6b22849a-2204-4970-b546-cd235cfec47c 1 2024-02-20T02:09:33.571 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.572+0000 7fedbf00bf00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-20T02:09:33.628 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.629+0000 7fedbf00bf00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-20T02:09:33.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.625+0000 7f2ad0e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2ad0e6d640 1 Processor -- start 2024-02-20T02:09:33.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2ad0e6d640 1 -- start start 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2ad0e6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f2acc112c90 0x7f2acc1130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2ad0e6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2acc113860 con 0x7f2acc117610 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2ad0e6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc118150 con 0x7f2acc112c90 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2aca59c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f2acc112c90 0x7f2acc1130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2aca59c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f2acc112c90 0x7f2acc1130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33798/0 (socket says 172.21.15.204:33798) 2024-02-20T02:09:33.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.631+0000 7f2aca59c640 1 -- 172.21.15.204:0/2705197215 learned_addr learned my addr 172.21.15.204:0/2705197215 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2aca59c640 1 -- 172.21.15.204:0/2705197215 >> v1:172.21.15.125:6789/0 conn(0x7f2acc117610 legacy=0x7f2acc117a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2aca59c640 1 -- 172.21.15.204:0/2705197215 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc1182c0 con 0x7f2acc112c90 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2aca59c640 1 --2- 172.21.15.204:0/2705197215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1130a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16837375c6151adb server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2ac9d9b640 1 -- 172.21.15.204:0/2705197215 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2abc009dd0 con 0x7f2acc112c90 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2ac9d9b640 1 -- 172.21.15.204:0/2705197215 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2abc00e230 con 0x7f2acc112c90 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.632+0000 7f2ac9d9b640 1 -- 172.21.15.204:0/2705197215 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2abc00e4d0 con 0x7f2acc112c90 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/2705197215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 msgr2=0x7f2acc1130a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 --2- 172.21.15.204:0/2705197215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1130a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/2705197215 shutdown_connections 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 --2- 172.21.15.204:0/2705197215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1130a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/2705197215 >> 172.21.15.204:0/2705197215 conn(0x7f2acc06d8a0 msgr2=0x7f2acc110dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/2705197215 shutdown_connections 2024-02-20T02:09:33.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/2705197215 wait complete. 2024-02-20T02:09:33.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.633+0000 7f2ad0e6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2ad0e6d640 1 Processor -- start 2024-02-20T02:09:33.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2ad0e6d640 1 -- start start 2024-02-20T02:09:33.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2ad0e6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2ad0e6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc1a7f50 con 0x7f2acc112c90 2024-02-20T02:09:33.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2aca59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2aca59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33804/0 (socket says 172.21.15.204:33804) 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.634+0000 7f2aca59c640 1 -- 172.21.15.204:0/1567600633 learned_addr learned my addr 172.21.15.204:0/1567600633 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2aca59c640 1 -- 172.21.15.204:0/1567600633 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc1a8090 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2aca59c640 1 --2- 172.21.15.204:0/1567600633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2abc014540 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2abc0146a0 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc1a82e0 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2abc014940 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc1ac290 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f2abc013400 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.635+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 862+0+0 (crc 0 0 0) 0x7f2abc019040 con 0x7f2acc112c90 2024-02-20T02:09:33.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.636+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc062a30 con 0x7f2acc112c90 2024-02-20T02:09:33.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.638+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f2abc013890 con 0x7f2acc112c90 2024-02-20T02:09:33.678 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:33.679+0000 7fedbf00bf00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-20T02:09:33.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.710+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b22849a-2204-4970-b546-cd235cfec47c", "id": 1} v 0) v1 -- 0x7f2acc066ef0 con 0x7f2acc112c90 2024-02-20T02:09:33.709 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.711+0000 7f2ac2ffd640 1 -- 172.21.15.204:0/1567600633 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b22849a-2204-4970-b546-cd235cfec47c", "id": 1}]=0 v5) v1 ==== 112+0+2 (crc 0 0 0) 0x7f2abc013a30 con 0x7f2acc112c90 2024-02-20T02:09:33.710 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 msgr2=0x7f2acc1a7980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 --2- 172.21.15.204:0/1567600633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 shutdown_connections 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 --2- 172.21.15.204:0/1567600633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2acc112c90 0x7f2acc1a7980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 >> 172.21.15.204:0/1567600633 conn(0x7f2acc06d8a0 msgr2=0x7f2acc111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 shutdown_connections 2024-02-20T02:09:33.711 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.712+0000 7f2ad0e6d640 1 -- 172.21.15.204:0/1567600633 wait complete. 2024-02-20T02:09:33.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new cc53d5a4-28f8-4fbd-a10a-d0293de49179 2 2024-02-20T02:09:33.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.823+0000 7f3c1b42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.829+0000 7f3c1b42d640 1 Processor -- start 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.829+0000 7f3c1b42d640 1 -- start start 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.829+0000 7f3c1b42d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3c14112c90 0x7f3c141130a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.829+0000 7f3c1b42d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c14113860 con 0x7f3c14117610 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.829+0000 7f3c1b42d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14118150 con 0x7f3c14112c90 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3c14112c90 0x7f3c141130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3c14112c90 0x7f3c141130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33816/0 (socket says 172.21.15.204:33816) 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c189c8640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f3c14117610 0x7f3c14117a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.204:42132/0 (socket says 172.21.15.204:42132) 2024-02-20T02:09:33.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 -- 172.21.15.204:0/3419961476 learned_addr learned my addr 172.21.15.204:0/3419961476 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 -- 172.21.15.204:0/3419961476 >> v1:172.21.15.125:6789/0 conn(0x7f3c14117610 legacy=0x7f3c14117a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c0bfff640 1 -- 172.21.15.204:0/3419961476 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141149622 0 0) 0x7f3c04001110 con 0x7f3c14117610 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 -- 172.21.15.204:0/3419961476 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c141182c0 con 0x7f3c14112c90 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c191c9640 1 --2- 172.21.15.204:0/3419961476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141130a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5319cacc3a6e7d server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c0bfff640 1 -- 172.21.15.204:0/3419961476 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3c10009dd0 con 0x7f3c14112c90 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c0bfff640 1 -- 172.21.15.204:0/3419961476 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3c1000e230 con 0x7f3c14112c90 2024-02-20T02:09:33.830 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.830+0000 7f3c0bfff640 1 -- 172.21.15.204:0/3419961476 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3c1000e4d0 con 0x7f3c14112c90 2024-02-20T02:09:33.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 -- 172.21.15.204:0/3419961476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 msgr2=0x7f3c141130a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.832 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 --2- 172.21.15.204:0/3419961476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141130a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 -- 172.21.15.204:0/3419961476 shutdown_connections 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 --2- 172.21.15.204:0/3419961476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141130a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 -- 172.21.15.204:0/3419961476 >> 172.21.15.204:0/3419961476 conn(0x7f3c1406d8a0 msgr2=0x7f3c14110dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 -- 172.21.15.204:0/3419961476 shutdown_connections 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 1 -- 172.21.15.204:0/3419961476 wait complete. 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.831+0000 7f3c1b42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c1b42d640 1 Processor -- start 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c1b42d640 1 -- start start 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c1b42d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c1b42d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a7f10 con 0x7f3c14112c90 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c191c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c191c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33818/0 (socket says 172.21.15.204:33818) 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.832+0000 7f3c191c9640 1 -- 172.21.15.204:0/1504882313 learned_addr learned my addr 172.21.15.204:0/1504882313 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:33.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c191c9640 1 -- 172.21.15.204:0/1504882313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c141a8050 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c191c9640 1 --2- 172.21.15.204:0/1504882313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3c10014620 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3c10014780 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3c10014a20 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c141a82a0 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f3c10013400 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.833+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c141ac250 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.834+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 977+0+0 (crc 0 0 0) 0x7f3c10019040 con 0x7f3c14112c90 2024-02-20T02:09:33.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.834+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c14062a30 con 0x7f3c14112c90 2024-02-20T02:09:33.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.837+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f3c10012390 con 0x7f3c14112c90 2024-02-20T02:09:33.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.909+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cc53d5a4-28f8-4fbd-a10a-d0293de49179", "id": 2} v 0) v1 -- 0x7f3c14066f50 con 0x7f3c14112c90 2024-02-20T02:09:33.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.911+0000 7f3c09ffb640 1 -- 172.21.15.204:0/1504882313 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc53d5a4-28f8-4fbd-a10a-d0293de49179", "id": 2}]=0 v6) v1 ==== 112+0+2 (crc 0 0 0) 0x7f3c100136b0 con 0x7f3c14112c90 2024-02-20T02:09:33.909 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-02-20T02:09:33.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 msgr2=0x7f3c141a7940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 --2- 172.21.15.204:0/1504882313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 shutdown_connections 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 --2- 172.21.15.204:0/1504882313 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c14112c90 0x7f3c141a7940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 >> 172.21.15.204:0/1504882313 conn(0x7f3c1406d8a0 msgr2=0x7f3c14111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 shutdown_connections 2024-02-20T02:09:33.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:33.912+0000 7f3c1b42d640 1 -- 172.21.15.204:0/1504882313 wait complete. 2024-02-20T02:09:33.921 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 523402ba-0344-4708-bb63-297d11afad8d 3 2024-02-20T02:09:34.028 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.024+0000 7f473d910640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:34.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.030+0000 7f473d910640 1 Processor -- start 2024-02-20T02:09:34.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.030+0000 7f473d910640 1 -- start start 2024-02-20T02:09:34.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.030+0000 7f473d910640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f4738115900 0x7f4738115d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:34.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.030+0000 7f473d910640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4738111550 con 0x7f4738110a30 2024-02-20T02:09:34.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.030+0000 7f473d910640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47381116e0 con 0x7f4738115900 2024-02-20T02:09:34.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f4738115900 0x7f4738115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f4738115900 0x7f4738115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33824/0 (socket says 172.21.15.204:33824) 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 -- 172.21.15.204:0/171507555 learned_addr learned my addr 172.21.15.204:0/171507555 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f472ffff640 1 --1- 172.21.15.204:0/171507555 >> v1:172.21.15.125:6789/0 conn(0x7f4738110a30 0x7f4738110e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.204:42134/0 (socket says 172.21.15.204:42134) 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 -- 172.21.15.204:0/171507555 >> v1:172.21.15.125:6789/0 conn(0x7f4738110a30 legacy=0x7f4738110e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 -- 172.21.15.204:0/171507555 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738111850 con 0x7f4738115900 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f4736ffd640 1 --2- 172.21.15.204:0/171507555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738115900 0x7f4738115d10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0443d79bf5f48a server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f47367fc640 1 -- 172.21.15.204:0/171507555 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4728009dd0 con 0x7f4738115900 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f47367fc640 1 -- 172.21.15.204:0/171507555 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f472800e230 con 0x7f4738115900 2024-02-20T02:09:34.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.031+0000 7f47367fc640 1 -- 172.21.15.204:0/171507555 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f472800e4d0 con 0x7f4738115900 2024-02-20T02:09:34.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 -- 172.21.15.204:0/171507555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738115900 msgr2=0x7f4738115d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 --2- 172.21.15.204:0/171507555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738115900 0x7f4738115d10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 -- 172.21.15.204:0/171507555 shutdown_connections 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 --2- 172.21.15.204:0/171507555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738115900 0x7f4738115d10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 -- 172.21.15.204:0/171507555 >> 172.21.15.204:0/171507555 conn(0x7f47380710b0 msgr2=0x7f473810edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 -- 172.21.15.204:0/171507555 shutdown_connections 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.032+0000 7f473d910640 1 -- 172.21.15.204:0/171507555 wait complete. 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.033+0000 7f473d910640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.033+0000 7f473d910640 1 Processor -- start 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.033+0000 7f473d910640 1 -- start start 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.033+0000 7f473d910640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.033+0000 7f473d910640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473806c610 con 0x7f4738110a30 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4736ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4736ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.204:33830/0 (socket says 172.21.15.204:33830) 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4736ffd640 1 -- 172.21.15.204:0/1286454517 learned_addr learned my addr 172.21.15.204:0/1286454517 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4736ffd640 1 -- 172.21.15.204:0/1286454517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473806c750 con 0x7f4738110a30 2024-02-20T02:09:34.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4736ffd640 1 --2- 172.21.15.204:0/1286454517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4728014540 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f47280146a0 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4728014940 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473806c940 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473806cd90 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f4728013400 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.034+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1092+0+0 (crc 0 0 0) 0x7f4728019040 con 0x7f4738110a30 2024-02-20T02:09:34.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.035+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738062a30 con 0x7f4738110a30 2024-02-20T02:09:34.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.037+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f47280125c0 con 0x7f4738110a30 2024-02-20T02:09:34.085 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.086+0000 7fedbf00bf00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-20T02:09:34.109 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.110+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd new", "uuid": "523402ba-0344-4708-bb63-297d11afad8d", "id": 3} v 0) v1 -- 0x7f4738066ef0 con 0x7f4738110a30 2024-02-20T02:09:34.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.111+0000 7f4734ff9640 1 -- 172.21.15.204:0/1286454517 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "523402ba-0344-4708-bb63-297d11afad8d", "id": 3}]=0 v7) v1 ==== 112+0+2 (crc 0 0 0) 0x7f4728012760 con 0x7f4738110a30 2024-02-20T02:09:34.110 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-02-20T02:09:34.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 msgr2=0x7f473806c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 --2- 172.21.15.204:0/1286454517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 shutdown_connections 2024-02-20T02:09:34.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 --2- 172.21.15.204:0/1286454517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4738110a30 0x7f473806c040 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 >> 172.21.15.204:0/1286454517 conn(0x7f47380710b0 msgr2=0x7f47381144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:34.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 shutdown_connections 2024-02-20T02:09:34.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-02-20T02:09:34.112+0000 7f473d910640 1 -- 172.21.15.204:0/1286454517 wait complete. 2024-02-20T02:09:34.121 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-02-20T02:09:34.159 INFO:tasks.ceph.osd.0:Restarting daemon 2024-02-20T02:09:34.159 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-02-20T02:09:34.202 INFO:tasks.ceph.osd.0:Started 2024-02-20T02:09:34.202 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-02-20T02:09:34.215 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.217+0000 7fedbf00bf00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-20T02:09:34.238 INFO:tasks.ceph.osd.1:Restarting daemon 2024-02-20T02:09:34.238 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-02-20T02:09:34.281 INFO:tasks.ceph.osd.1:Started 2024-02-20T02:09:34.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-02-20T02:09:34.314 INFO:tasks.ceph.osd.2:Restarting daemon 2024-02-20T02:09:34.314 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-02-20T02:09:34.344 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.345+0000 7fedbf00bf00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-20T02:09:34.346 INFO:tasks.ceph.osd.2:Started 2024-02-20T02:09:34.346 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-02-20T02:09:34.377 INFO:tasks.ceph.osd.3:Restarting daemon 2024-02-20T02:09:34.377 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-02-20T02:09:34.395 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.396+0000 7fedbf00bf00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-20T02:09:34.397 INFO:tasks.ceph.osd.3:Started 2024-02-20T02:09:34.397 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-20T02:09:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.516+0000 7f21aad1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.522+0000 7f21aad1d640 1 Processor -- start 2024-02-20T02:09:34.520 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.522+0000 7f21aad1d640 1 -- start start 2024-02-20T02:09:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.522+0000 7f21aad1d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f21a4137050 0x7f21a4139470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.522+0000 7f21aad1d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a409a810 con 0x7f21a41342a0 2024-02-20T02:09:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.522+0000 7f21aad1d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a4099f20 con 0x7f21a4137050 2024-02-20T02:09:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a3fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f21a4137050 0x7f21a4139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a8ab9640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f21a41342a0 0x7f21a41346b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36240/0 (socket says 172.21.15.125:36240) 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a8ab9640 1 -- 172.21.15.125:0/3290712708 learned_addr learned my addr 172.21.15.125:0/3290712708 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a3fff640 1 -- 172.21.15.125:0/3290712708 >> v1:172.21.15.125:6789/0 conn(0x7f21a41342a0 legacy=0x7f21a41346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a3fff640 1 -- 172.21.15.125:0/3290712708 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a409bbe0 con 0x7f21a4137050 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a3fff640 1 --2- 172.21.15.125:0/3290712708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a4137050 0x7f21a4139470 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9b54a58d38fed server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a37fe640 1 -- 172.21.15.125:0/3290712708 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2190009cd0 con 0x7f21a4137050 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a37fe640 1 -- 172.21.15.125:0/3290712708 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2190009e30 con 0x7f21a4137050 2024-02-20T02:09:34.523 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.523+0000 7f21a37fe640 1 -- 172.21.15.125:0/3290712708 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f219000e360 con 0x7f21a4137050 2024-02-20T02:09:34.524 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 -- 172.21.15.125:0/3290712708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a4137050 msgr2=0x7f21a4139470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 --2- 172.21.15.125:0/3290712708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a4137050 0x7f21a4139470 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 -- 172.21.15.125:0/3290712708 shutdown_connections 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 --2- 172.21.15.125:0/3290712708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a4137050 0x7f21a4139470 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 -- 172.21.15.125:0/3290712708 >> 172.21.15.125:0/3290712708 conn(0x7f21a406c8a0 msgr2=0x7f21a406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 -- 172.21.15.125:0/3290712708 shutdown_connections 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.524+0000 7f21aad1d640 1 -- 172.21.15.125:0/3290712708 wait complete. 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.525+0000 7f21aad1d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.526+0000 7f21aad1d640 1 Processor -- start 2024-02-20T02:09:34.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.526+0000 7f21aad1d640 1 -- start start 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21aad1d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21aad1d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4121850 con 0x7f21a41342a0 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a8ab9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a8ab9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37168/0 (socket says 172.21.15.125:37168) 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a8ab9640 1 -- 172.21.15.125:0/3581942063 learned_addr learned my addr 172.21.15.125:0/3581942063 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:34.527 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a8ab9640 1 -- 172.21.15.125:0/3581942063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a41205e0 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a8ab9640 1 --2- 172.21.15.125:0/3581942063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.527+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f21980059d0 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2198005b30 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a4121ce0 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2198005dd0 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a41312b0 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f2198007440 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.528+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f219800fb10 con 0x7f21a41342a0 2024-02-20T02:09:34.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.529+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4120990 con 0x7f21a41342a0 2024-02-20T02:09:34.529 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.528+0000 7fedbf00bf00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-20T02:09:34.530 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.531+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f219800fd70 con 0x7f21a41342a0 2024-02-20T02:09:34.593 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.594+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21a4121a90 con 0x7f21a41342a0 2024-02-20T02:09:34.594 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.595+0000 7f21a17fa640 1 -- 172.21.15.125:0/3581942063 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7f219800f5f0 con 0x7f21a41342a0 2024-02-20T02:09:34.594 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:34.594 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":7,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:09:34.112255+0000","last_up_change":"0.000000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 msgr2=0x7f21a411e210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 --2- 172.21.15.125:0/3581942063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 shutdown_connections 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 --2- 172.21.15.125:0/3581942063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21a41342a0 0x7f21a411e210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 >> 172.21.15.125:0/3581942063 conn(0x7f21a406c8a0 msgr2=0x7f21a4131b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 shutdown_connections 2024-02-20T02:09:34.595 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:34.596+0000 7f21aad1d640 1 -- 172.21.15.125:0/3581942063 wait complete. 2024-02-20T02:09:34.607 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-02-20T02:09:34.607 INFO:tasks.ceph:Waiting for OSDs to come up 2024-02-20T02:09:34.645 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.646+0000 7fedbf00bf00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-20T02:09:34.829 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.830+0000 7fedbf00bf00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-20T02:09:34.878 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:34.879+0000 7fedbf00bf00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-20T02:09:35.010 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-20T02:09:35.042 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:35.043+0000 7fedbf00bf00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-20T02:09:35.108 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.106+0000 7f3e1102d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:35.109 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.110+0000 7f3e1102d640 1 Processor -- start 2024-02-20T02:09:35.109 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e1102d640 1 -- start start 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e1102d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e1102d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c09f210 con 0x7f3e0c09e460 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e1102d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c09eaf0 con 0x7f3e0c0a0fd0 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e03fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e0ad9d640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f3e0c09e460 0x7f3e0c0a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36246/0 (socket says 172.21.15.125:36246) 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e03fff640 1 --2- 172.21.15.125:0/1618977367 >> v2:172.21.15.125:3300/0 conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37170/0 (socket says 172.21.15.125:37170) 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.111+0000 7f3e0ad9d640 1 -- 172.21.15.125:0/1618977367 learned_addr learned my addr 172.21.15.125:0/1618977367 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e03fff640 1 -- 172.21.15.125:0/1618977367 >> v1:172.21.15.125:6789/0 conn(0x7f3e0c09e460 legacy=0x7f3e0c0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e03fff640 1 -- 172.21.15.125:0/1618977367 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c09d990 con 0x7f3e0c0a0fd0 2024-02-20T02:09:35.111 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e0a59c640 1 -- 172.21.15.125:0/1618977367 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112951951 0 0) 0x7f3e0c09f210 con 0x7f3e0c09e460 2024-02-20T02:09:35.112 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e03fff640 1 --2- 172.21.15.125:0/1618977367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fed468785c3d7d server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:35.112 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e0a59c640 1 -- 172.21.15.125:0/1618977367 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3df4009ce0 con 0x7f3e0c0a0fd0 2024-02-20T02:09:35.112 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e0a59c640 1 -- 172.21.15.125:0/1618977367 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3df4009e40 con 0x7f3e0c0a0fd0 2024-02-20T02:09:35.112 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.112+0000 7f3e0a59c640 1 -- 172.21.15.125:0/1618977367 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3df400e360 con 0x7f3e0c0a0fd0 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 -- 172.21.15.125:0/1618977367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c0a0fd0 msgr2=0x7f3e0c1396c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 --2- 172.21.15.125:0/1618977367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 -- 172.21.15.125:0/1618977367 shutdown_connections 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 --2- 172.21.15.125:0/1618977367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c0a0fd0 0x7f3e0c1396c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 -- 172.21.15.125:0/1618977367 >> 172.21.15.125:0/1618977367 conn(0x7f3e0c0a2b60 msgr2=0x7f3e0c0a2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 -- 172.21.15.125:0/1618977367 shutdown_connections 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 1 -- 172.21.15.125:0/1618977367 wait complete. 2024-02-20T02:09:35.113 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.113+0000 7f3e1102d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:35.114 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.114+0000 7f3e1102d640 1 Processor -- start 2024-02-20T02:09:35.114 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.114+0000 7f3e1102d640 1 -- start start 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e1102d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e1102d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c1c5140 con 0x7f3e0c09e460 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:37184/0 (socket says 172.21.15.125:37184) 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e0ad9d640 1 -- 172.21.15.125:0/113417620 learned_addr learned my addr 172.21.15.125:0/113417620 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:35.115 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e0ad9d640 1 -- 172.21.15.125:0/113417620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c1aae30 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.115+0000 7f3e0ad9d640 1 --2- 172.21.15.125:0/113417620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3dfc004af0 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3dfc004c50 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3dfc007120 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c1c5370 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c1ab490 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f3dfc00c4e0 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.116+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f3dfc00fbb0 con 0x7f3e0c09e460 2024-02-20T02:09:35.116 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.117+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c062a30 con 0x7f3e0c09e460 2024-02-20T02:09:35.117 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.119+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f3dfc00f230 con 0x7f3e0c09e460 2024-02-20T02:09:35.179 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.181+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e0c067390 con 0x7f3e0c09e460 2024-02-20T02:09:35.180 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.181+0000 7f3e037fe640 1 -- 172.21.15.125:0/113417620 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7f3dfc00fe10 con 0x7f3e0c09e460 2024-02-20T02:09:35.180 INFO:teuthology.misc.health.smithi125.stdout: 2024-02-20T02:09:35.180 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":7,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:09:34.112255+0000","last_up_change":"0.000000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 msgr2=0x7f3e0c1c5680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 --2- 172.21.15.125:0/113417620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 shutdown_connections 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 --2- 172.21.15.125:0/113417620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3e0c09e460 0x7f3e0c1c5680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 >> 172.21.15.125:0/113417620 conn(0x7f3e0c0a2b60 msgr2=0x7f3e0c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 shutdown_connections 2024-02-20T02:09:35.181 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:35.182+0000 7f3e1102d640 1 -- 172.21.15.125:0/113417620 wait complete. 2024-02-20T02:09:35.190 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-02-20T02:09:35.935 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:35.937+0000 7fedbf00bf00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-20T02:09:36.230 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:36.231+0000 7fedbf00bf00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-20T02:09:36.288 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:36.289+0000 7fedbf00bf00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-20T02:09:36.411 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:36.412+0000 7fedbf00bf00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-20T02:09:37.432 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-02-20T02:09:37.433+0000 7fed5eba2640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-02-20T02:09:41.594 DEBUG:teuthology.orchestra.run.smithi125:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-20T02:09:41.697 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.695+0000 7fefaae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:41.697 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.699+0000 7fefaae6d640 1 Processor -- start 2024-02-20T02:09:41.698 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.699+0000 7fefaae6d640 1 -- start start 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefaae6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fefa4099370 0x7fefa4130d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefaae6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa4131650 con 0x7fefa4131240 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefaae6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40998f0 con 0x7fefa4099370 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa8c09640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fefa4099370 0x7fefa4130d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa3fff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fefa4131240 0x7fefa412d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36192/0 (socket says 172.21.15.125:36192) 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa3fff640 1 -- 172.21.15.125:0/3159074666 learned_addr learned my addr 172.21.15.125:0/3159074666 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:41.699 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa8c09640 1 -- 172.21.15.125:0/3159074666 >> v1:172.21.15.125:6789/0 conn(0x7fefa4131240 legacy=0x7fefa412d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa37fe640 1 -- 172.21.15.125:0/3159074666 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185400539 0 0) 0x7fefa4131650 con 0x7fefa4131240 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa8c09640 1 -- 172.21.15.125:0/3159074666 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa406dc10 con 0x7fefa4099370 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa8c09640 1 --2- 172.21.15.125:0/3159074666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4130d00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de34df2949851f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa37fe640 1 -- 172.21.15.125:0/3159074666 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fef98009ce0 con 0x7fefa4099370 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa37fe640 1 -- 172.21.15.125:0/3159074666 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fef98009e40 con 0x7fefa4099370 2024-02-20T02:09:41.700 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.700+0000 7fefa37fe640 1 -- 172.21.15.125:0/3159074666 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fef9800e360 con 0x7fefa4099370 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 -- 172.21.15.125:0/3159074666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 msgr2=0x7fefa4130d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3159074666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4130d00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 -- 172.21.15.125:0/3159074666 shutdown_connections 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3159074666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4130d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 -- 172.21.15.125:0/3159074666 >> 172.21.15.125:0/3159074666 conn(0x7fefa409e770 msgr2=0x7fefa409eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 -- 172.21.15.125:0/3159074666 shutdown_connections 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 1 -- 172.21.15.125:0/3159074666 wait complete. 2024-02-20T02:09:41.701 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.701+0000 7fefaae6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:41.702 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.702+0000 7fefaae6d640 1 Processor -- start 2024-02-20T02:09:41.702 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.702+0000 7fefaae6d640 1 -- start start 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.703+0000 7fefaae6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.703+0000 7fefaae6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4072f80 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.703+0000 7fefa8c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.703+0000 7fefa8c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53340/0 (socket says 172.21.15.125:53340) 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.703+0000 7fefa8c09640 1 -- 172.21.15.125:0/3380742734 learned_addr learned my addr 172.21.15.125:0/3380742734 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefa8c09640 1 -- 172.21.15.125:0/3380742734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa409a590 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefa8c09640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fef98014410 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fef98014570 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fef98014810 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa4129f50 con 0x7fefa4099370 2024-02-20T02:09:41.704 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.704+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa412a5d0 con 0x7fefa4099370 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.705+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 4) v1 ==== 45152+0+0 (crc 0 0 0) 0x7fef980149c0 con 0x7fefa4099370 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.705+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa412a820 con 0x7fefa4099370 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.705+0000 7fefa17fa640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 0x7fef8803a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.705+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2319+0+0 (crc 0 0 0) 0x7fef98019040 con 0x7fefa4099370 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.705+0000 7fefa3fff640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 0x7fef8803a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:41.705 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.706+0000 7fefa3fff640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 0x7fef8803a360 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:41.709 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.710+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7fef9800f7b0 con 0x7fefa4099370 2024-02-20T02:09:41.885 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.886+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa4062a30 con 0x7fefa4099370 2024-02-20T02:09:41.885 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.887+0000 7fefa17fa640 1 -- 172.21.15.125:0/3380742734 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6940 (crc 0 0 0) 0x7fef9800f7b0 con 0x7fefa4099370 2024-02-20T02:09:41.886 INFO:teuthology.misc.health.smithi125.stdout: 2024-02-20T02:09:41.886 INFO:teuthology.misc.health.smithi125.stdout:{"epoch":12,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:09:41.143873+0000","last_up_change":"2024-02-20T02:09:39.134569+0000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-20T02:09:38.470855+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3754305691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":3754305691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6809","nonce":3754305691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3754305691}]},"public_addr":"172.21.15.125:6802/3754305691","cluster_addr":"172.21.15.125:6801/3754305691","heartbeat_back_addr":"172.21.15.125:6809/3754305691","heartbeat_front_addr":"172.21.15.125:6810/3754305691","state":["exists","up"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3877308034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":3877308034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3877308034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":3877308034}]},"public_addr":"172.21.15.125:6804/3877308034","cluster_addr":"172.21.15.125:6805/3877308034","heartbeat_back_addr":"172.21.15.125:6806/3877308034","heartbeat_front_addr":"172.21.15.125:6803/3877308034","state":["exists","up"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":774159808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":774159808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":774159808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":774159808}]},"public_addr":"172.21.15.204:6811/774159808","cluster_addr":"172.21.15.204:6812/774159808","heartbeat_back_addr":"172.21.15.204:6808/774159808","heartbeat_front_addr":"172.21.15.204:6807/774159808","state":["exists","up"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":409650538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":409650538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":409650538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":409650538}]},"public_addr":"172.21.15.204:6804/409650538","cluster_addr":"172.21.15.204:6803/409650538","heartbeat_back_addr":"172.21.15.204:6810/409650538","heartbeat_front_addr":"172.21.15.204:6809/409650538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 msgr2=0x7fef8803a360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 0x7fef8803a360 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 msgr2=0x7fefa4128af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 shutdown_connections 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fef88037e50 0x7fef8803a360 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 --2- 172.21.15.125:0/3380742734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fefa4099370 0x7fefa4128af0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 >> 172.21.15.125:0/3380742734 conn(0x7fefa409e770 msgr2=0x7fefa41b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 shutdown_connections 2024-02-20T02:09:41.888 INFO:teuthology.misc.health.smithi125.stderr:2024-02-20T02:09:41.889+0000 7fefaae6d640 1 -- 172.21.15.125:0/3380742734 wait complete. 2024-02-20T02:09:41.897 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-02-20T02:09:41.897 INFO:tasks.ceph:Creating RBD pool 2024-02-20T02:09:41.897 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd pool create rbd 8 2024-02-20T02:09:42.003 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:41.998+0000 7f3bbee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:42.003 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.005+0000 7f3bbee6d640 1 Processor -- start 2024-02-20T02:09:42.003 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.005+0000 7f3bbee6d640 1 -- start start 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.005+0000 7f3bbee6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3bb809a5b0 0x7f3bb8136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bbee6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb8099a70 con 0x7f3bb809bc40 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bbee6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb809a9c0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb7fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3bb809a5b0 0x7f3bb8136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bbcc09640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f3bb809bc40 0x7f3bb8099ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36202/0 (socket says 172.21.15.125:36202) 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bbcc09640 1 -- 172.21.15.125:0/3083124912 learned_addr learned my addr 172.21.15.125:0/3083124912 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb7fff640 1 -- 172.21.15.125:0/3083124912 >> v1:172.21.15.125:6789/0 conn(0x7f3bb809bc40 legacy=0x7f3bb8099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:42.005 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb7fff640 1 -- 172.21.15.125:0/3083124912 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb809ae60 con 0x7f3bb809a5b0 2024-02-20T02:09:42.006 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb7fff640 1 --2- 172.21.15.125:0/3083124912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb8136bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fbcecafd9e633b server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:42.006 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb77fe640 1 -- 172.21.15.125:0/3083124912 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ba4009ce0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.006 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb77fe640 1 -- 172.21.15.125:0/3083124912 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3ba4009e40 con 0x7f3bb809a5b0 2024-02-20T02:09:42.006 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.006+0000 7f3bb77fe640 1 -- 172.21.15.125:0/3083124912 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ba400e360 con 0x7f3bb809a5b0 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 -- 172.21.15.125:0/3083124912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 msgr2=0x7f3bb8136bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/3083124912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb8136bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 -- 172.21.15.125:0/3083124912 shutdown_connections 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/3083124912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb8136bb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 -- 172.21.15.125:0/3083124912 >> 172.21.15.125:0/3083124912 conn(0x7f3bb809fae0 msgr2=0x7f3bb809fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 -- 172.21.15.125:0/3083124912 shutdown_connections 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 1 -- 172.21.15.125:0/3083124912 wait complete. 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.007+0000 7f3bbee6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.008+0000 7f3bbee6d640 1 Processor -- start 2024-02-20T02:09:42.007 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.008+0000 7f3bbee6d640 1 -- start start 2024-02-20T02:09:42.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbee6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:42.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbee6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb811b0e0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbcc09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:42.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbcc09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53370/0 (socket says 172.21.15.125:53370) 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbcc09640 1 -- 172.21.15.125:0/2598575939 learned_addr learned my addr 172.21.15.125:0/2598575939 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbcc09640 1 -- 172.21.15.125:0/2598575939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb811c3d0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbcc09640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3bac0059d0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3bac005b30 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb811dc50 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3bac005dd0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.009+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb81c5410 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.010+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 4) v1 ==== 45152+0+0 (crc 0 0 0) 0x7f3bac0070b0 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.010+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb8062a30 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.010+0000 7f3bb57fa640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 0x7f3b9c03a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.010+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f3bac047190 con 0x7f3bb809a5b0 2024-02-20T02:09:42.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.011+0000 7f3bb7fff640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 0x7f3b9c03a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:42.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.011+0000 7f3bb7fff640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 0x7f3b9c03a360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:42.014 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.015+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f3bac00d510 con 0x7f3bb809a5b0 2024-02-20T02:09:42.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:42.188+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3bb8068410 con 0x7f3bb809a5b0 2024-02-20T02:09:43.149 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.150+0000 7f3bb57fa640 1 -- 172.21.15.125:0/2598575939 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (crc 0 0 0) 0x7f3bac00d150 con 0x7f3bb809a5b0 2024-02-20T02:09:43.149 INFO:teuthology.orchestra.run.smithi125.stderr:pool 'rbd' created 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 msgr2=0x7f3b9c03a360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 0x7f3b9c03a360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 msgr2=0x7f3bb81c5a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 shutdown_connections 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b9c037e50 0x7f3b9c03a360 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 --2- 172.21.15.125:0/2598575939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bb809a5b0 0x7f3bb81c5a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 >> 172.21.15.125:0/2598575939 conn(0x7f3bb809fae0 msgr2=0x7f3bb8070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:43.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 shutdown_connections 2024-02-20T02:09:43.152 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.152+0000 7f3bbee6d640 1 -- 172.21.15.125:0/2598575939 wait complete. 2024-02-20T02:09:43.166 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-02-20T02:09:43.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.270+0000 7f0df142d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:43.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.279+0000 7f0df142d640 1 Processor -- start 2024-02-20T02:09:43.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.279+0000 7f0df142d640 1 -- start start 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0df142d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0dec132420 0x7f0dec136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0df142d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec09fa60 con 0x7f0dec131900 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0df142d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec0712d0 con 0x7f0dec132420 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0dea7fc640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0dec132420 0x7f0dec136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0deaffd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f0dec131900 0x7f0dec131d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36216/0 (socket says 172.21.15.125:36216) 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0deaffd640 1 -- 172.21.15.125:0/971651922 learned_addr learned my addr 172.21.15.125:0/971651922 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0dea7fc640 1 -- 172.21.15.125:0/971651922 >> v1:172.21.15.125:6789/0 conn(0x7f0dec131900 legacy=0x7f0dec131d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0de9ffb640 1 -- 172.21.15.125:0/971651922 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590730355 0 0) 0x7f0dec09fa60 con 0x7f0dec131900 2024-02-20T02:09:43.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0dea7fc640 1 -- 172.21.15.125:0/971651922 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec09fc90 con 0x7f0dec132420 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0dea7fc640 1 --2- 172.21.15.125:0/971651922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec132420 0x7f0dec136ea0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a3c1554d466421 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0de9ffb640 1 -- 172.21.15.125:0/971651922 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0de0009ce0 con 0x7f0dec132420 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0de9ffb640 1 -- 172.21.15.125:0/971651922 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0de0009e40 con 0x7f0dec132420 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.280+0000 7f0de9ffb640 1 -- 172.21.15.125:0/971651922 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0de000e360 con 0x7f0dec132420 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 -- 172.21.15.125:0/971651922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec132420 msgr2=0x7f0dec136ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 --2- 172.21.15.125:0/971651922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec132420 0x7f0dec136ea0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 -- 172.21.15.125:0/971651922 shutdown_connections 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 --2- 172.21.15.125:0/971651922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec132420 0x7f0dec136ea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 -- 172.21.15.125:0/971651922 >> 172.21.15.125:0/971651922 conn(0x7f0dec0a0c70 msgr2=0x7f0dec0a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 -- 172.21.15.125:0/971651922 shutdown_connections 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 1 -- 172.21.15.125:0/971651922 wait complete. 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.281+0000 7f0df142d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:43.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.282+0000 7f0df142d640 1 Processor -- start 2024-02-20T02:09:43.283 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.282+0000 7f0df142d640 1 -- start start 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0df142d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0df142d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec12de10 con 0x7f0dec131900 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0deaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0deaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53398/0 (socket says 172.21.15.125:53398) 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0deaffd640 1 -- 172.21.15.125:0/156814951 learned_addr learned my addr 172.21.15.125:0/156814951 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0deaffd640 1 -- 172.21.15.125:0/156814951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec11c380 con 0x7f0dec131900 2024-02-20T02:09:43.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.283+0000 7f0deaffd640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.284+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0ddc004e70 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.284+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0ddc00c870 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.284+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec11d9d0 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.284+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0ddc00cb10 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.284+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec11d260 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.285+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f0ddc00ccc0 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.285+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec11dbc0 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.285+0000 7f0dd37fe640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 0x7f0dd803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.285+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2661+0+0 (crc 0 0 0) 0x7f0ddc0478f0 con 0x7f0dec131900 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.285+0000 7f0dea7fc640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 0x7f0dd803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:43.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.286+0000 7f0dea7fc640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 0x7f0dd803a3f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:43.288 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.290+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f0ddc00a7b0 con 0x7f0dec131900 2024-02-20T02:09:43.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:43.462+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0dec11c640 con 0x7f0dec131900 2024-02-20T02:09:44.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.155+0000 7f0dd37fe640 1 -- 172.21.15.125:0/156814951 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (crc 0 0 0) 0x7f0ddc006550 con 0x7f0dec131900 2024-02-20T02:09:44.155 INFO:teuthology.orchestra.run.smithi125.stderr:enabled application 'rbd' on pool 'rbd' 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 msgr2=0x7f0dd803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 0x7f0dd803a3f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 msgr2=0x7f0dec11a250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 shutdown_connections 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0dd8037ee0 0x7f0dd803a3f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 --2- 172.21.15.125:0/156814951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0dec131900 0x7f0dec11a250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 >> 172.21.15.125:0/156814951 conn(0x7f0dec0a0c70 msgr2=0x7f0dec1b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 shutdown_connections 2024-02-20T02:09:44.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.158+0000 7f0df142d640 1 -- 172.21.15.125:0/156814951 wait complete. 2024-02-20T02:09:44.177 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-02-20T02:09:44.177 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-20T02:09:44.177 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-02-20T02:09:44.177 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-02-20T02:09:44.178 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-02-20T02:09:44.295 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.293+0000 7f589d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.297+0000 7f589d39d640 1 Processor -- start 2024-02-20T02:09:44.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.297+0000 7f589d39d640 1 -- start start 2024-02-20T02:09:44.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f589d39d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5898137050 0x7f5898139470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f589d39d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589809a810 con 0x7f58981342a0 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f589d39d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5898099f20 con 0x7f5898137050 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f58967fc640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5898137050 0x7f5898139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f5896ffd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f58981342a0 0x7f58981346b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36222/0 (socket says 172.21.15.125:36222) 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.298+0000 7f5896ffd640 1 -- 172.21.15.125:0/2817687005 learned_addr learned my addr 172.21.15.125:0/2817687005 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f58967fc640 1 -- 172.21.15.125:0/2817687005 >> v1:172.21.15.125:6789/0 conn(0x7f58981342a0 legacy=0x7f58981346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f58967fc640 1 -- 172.21.15.125:0/2817687005 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589809bbe0 con 0x7f5898137050 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f5895ffb640 1 -- 172.21.15.125:0/2817687005 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918090491 0 0) 0x7f589809a810 con 0x7f58981342a0 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f58967fc640 1 --2- 172.21.15.125:0/2817687005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5898137050 0x7f5898139470 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6155ebd91b4a04 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f5895ffb640 1 -- 172.21.15.125:0/2817687005 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5884009cd0 con 0x7f5898137050 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f5895ffb640 1 -- 172.21.15.125:0/2817687005 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5884009e30 con 0x7f5898137050 2024-02-20T02:09:44.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.299+0000 7f5895ffb640 1 -- 172.21.15.125:0/2817687005 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f588400e360 con 0x7f5898137050 2024-02-20T02:09:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 -- 172.21.15.125:0/2817687005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5898137050 msgr2=0x7f5898139470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 --2- 172.21.15.125:0/2817687005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5898137050 0x7f5898139470 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 -- 172.21.15.125:0/2817687005 shutdown_connections 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 --2- 172.21.15.125:0/2817687005 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5898137050 0x7f5898139470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 -- 172.21.15.125:0/2817687005 >> 172.21.15.125:0/2817687005 conn(0x7f589806c8a0 msgr2=0x7f589806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 -- 172.21.15.125:0/2817687005 shutdown_connections 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 1 -- 172.21.15.125:0/2817687005 wait complete. 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.300+0000 7f589d39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.301+0000 7f589d39d640 1 Processor -- start 2024-02-20T02:09:44.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.301+0000 7f589d39d640 1 -- start start 2024-02-20T02:09:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f589d39d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f589d39d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898121190 con 0x7f58981342a0 2024-02-20T02:09:44.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f5896ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f5896ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53418/0 (socket says 172.21.15.125:53418) 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f5896ffd640 1 -- 172.21.15.125:0/4179270876 learned_addr learned my addr 172.21.15.125:0/4179270876 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f5896ffd640 1 -- 172.21.15.125:0/4179270876 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589811e470 con 0x7f58981342a0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f5896ffd640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5890004af0 con 0x7f58981342a0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5890004c50 con 0x7f58981342a0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5890007120 con 0x7f58981342a0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898120ef0 con 0x7f58981342a0 2024-02-20T02:09:44.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.302+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58981219a0 con 0x7f58981342a0 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.303+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f5890007320 con 0x7f58981342a0 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.303+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589811e210 con 0x7f58981342a0 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.303+0000 7f588f7fe640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 0x7f588003a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.303+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f58900475a0 con 0x7f58981342a0 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.304+0000 7f58967fc640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 0x7f588003a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.304+0000 7f58967fc640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 0x7f588003a3a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.308+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f589000a9e0 con 0x7f58981342a0 2024-02-20T02:09:44.525 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.526+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5898062a30 con 0x7f58981342a0 2024-02-20T02:09:44.526 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.528+0000 7f588f7fe640 1 -- 172.21.15.125:0/4179270876 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86634 (crc 0 0 0) 0x7f589000a5d0 con 0x7f58981342a0 2024-02-20T02:09:44.527 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 msgr2=0x7f588003a3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 0x7f588003a3a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 msgr2=0x7f589811e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 shutdown_connections 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5880037e90 0x7f588003a3a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 --2- 172.21.15.125:0/4179270876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58981342a0 0x7f589811e790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 >> 172.21.15.125:0/4179270876 conn(0x7f589806c8a0 msgr2=0x7f5898131b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 shutdown_connections 2024-02-20T02:09:44.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.530+0000 7f589d39d640 1 -- 172.21.15.125:0/4179270876 wait complete. 2024-02-20T02:09:44.539 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1051662976},{"type":"v1","addr":"172.21.15.204:6801","nonce":1051662976}]},"active_addr":"172.21.15.204:6801/1051662976","active_change":"2024-02-20T02:09:36.426873+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":569685339}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":3586015487}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2565112529}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":1207572631}]}]} 2024-02-20T02:09:44.541 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-02-20T02:09:44.541 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-02-20T02:09:44.541 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-20T02:09:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.645+0000 7fe10b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe10b06d640 1 Processor -- start 2024-02-20T02:09:44.647 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe10b06d640 1 -- start start 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe10b06d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fe10412d4a0 0x7fe104135270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe10b06d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10409b710 con 0x7fe10409bea0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe10b06d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10406eb50 con 0x7fe10412d4a0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.649+0000 7fe103fff640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fe10412d4a0 0x7fe104135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe108e09640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fe10409bea0 0x7fe10412cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36224/0 (socket says 172.21.15.125:36224) 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe108e09640 1 -- 172.21.15.125:0/3157600024 learned_addr learned my addr 172.21.15.125:0/3157600024 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe103fff640 1 -- 172.21.15.125:0/3157600024 >> v1:172.21.15.125:6789/0 conn(0x7fe10409bea0 legacy=0x7fe10412cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe103fff640 1 -- 172.21.15.125:0/3157600024 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10406e070 con 0x7fe10412d4a0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe1037fe640 1 -- 172.21.15.125:0/3157600024 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668828920 0 0) 0x7fe10409b710 con 0x7fe10409bea0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe103fff640 1 --2- 172.21.15.125:0/3157600024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10412d4a0 0x7fe104135270 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeefc87d4505da8 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe1037fe640 1 -- 172.21.15.125:0/3157600024 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe0ec009ce0 con 0x7fe10412d4a0 2024-02-20T02:09:44.649 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe1037fe640 1 -- 172.21.15.125:0/3157600024 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe0ec009e40 con 0x7fe10412d4a0 2024-02-20T02:09:44.650 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.650+0000 7fe1037fe640 1 -- 172.21.15.125:0/3157600024 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe0ec00e360 con 0x7fe10412d4a0 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 -- 172.21.15.125:0/3157600024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10412d4a0 msgr2=0x7fe104135270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 --2- 172.21.15.125:0/3157600024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10412d4a0 0x7fe104135270 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 -- 172.21.15.125:0/3157600024 shutdown_connections 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 --2- 172.21.15.125:0/3157600024 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10412d4a0 0x7fe104135270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 -- 172.21.15.125:0/3157600024 >> 172.21.15.125:0/3157600024 conn(0x7fe10409fae0 msgr2=0x7fe10409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 -- 172.21.15.125:0/3157600024 shutdown_connections 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 1 -- 172.21.15.125:0/3157600024 wait complete. 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.651+0000 7fe10b06d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.652+0000 7fe10b06d640 1 Processor -- start 2024-02-20T02:09:44.651 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.652+0000 7fe10b06d640 1 -- start start 2024-02-20T02:09:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe10b06d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe10b06d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe104072be0 con 0x7fe10409bea0 2024-02-20T02:09:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe108e09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.653 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe108e09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53442/0 (socket says 172.21.15.125:53442) 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe108e09640 1 -- 172.21.15.125:0/351080979 learned_addr learned my addr 172.21.15.125:0/351080979 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe108e09640 1 -- 172.21.15.125:0/351080979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104072940 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe108e09640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe0fc004e70 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe0fc00c830 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe104071430 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe0fc00cad0 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.653+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1040721f0 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.654+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fe0fc00cc80 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.654+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe104062a30 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.654+0000 7fe1017fa640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 0x7fe0e403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.654+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe0fc0478a0 con 0x7fe10409bea0 2024-02-20T02:09:44.654 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.655+0000 7fe103fff640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 0x7fe0e403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.655 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.655+0000 7fe103fff640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 0x7fe0e403a3f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.657 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.659+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7fe0fc00a7b0 con 0x7fe10409bea0 2024-02-20T02:09:44.828 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.829+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe104068410 con 0x7fe10409bea0 2024-02-20T02:09:44.831 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.830+0000 7fe1017fa640 1 -- 172.21.15.125:0/351080979 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8741 (crc 0 0 0) 0x7fe0fc00a0c0 con 0x7fe10409bea0 2024-02-20T02:09:44.832 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:44.832 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":14,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:09:44.151306+0000","last_up_change":"2024-02-20T02:09:39.134569+0000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-20T02:09:38.470855+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-02-20T02:09:42.206626+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3754305691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":3754305691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6809","nonce":3754305691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3754305691}]},"public_addr":"172.21.15.125:6802/3754305691","cluster_addr":"172.21.15.125:6801/3754305691","heartbeat_back_addr":"172.21.15.125:6809/3754305691","heartbeat_front_addr":"172.21.15.125:6810/3754305691","state":["exists","up"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3877308034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":3877308034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3877308034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":3877308034}]},"public_addr":"172.21.15.125:6804/3877308034","cluster_addr":"172.21.15.125:6805/3877308034","heartbeat_back_addr":"172.21.15.125:6806/3877308034","heartbeat_front_addr":"172.21.15.125:6803/3877308034","state":["exists","up"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":774159808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":774159808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":774159808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":774159808}]},"public_addr":"172.21.15.204:6811/774159808","cluster_addr":"172.21.15.204:6812/774159808","heartbeat_back_addr":"172.21.15.204:6808/774159808","heartbeat_front_addr":"172.21.15.204:6807/774159808","state":["exists","up"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":409650538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":409650538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":409650538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":409650538}]},"public_addr":"172.21.15.204:6804/409650538","cluster_addr":"172.21.15.204:6803/409650538","heartbeat_back_addr":"172.21.15.204:6810/409650538","heartbeat_front_addr":"172.21.15.204:6809/409650538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 msgr2=0x7fe0e403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 0x7fe0e403a3f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 msgr2=0x7fe104129c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 shutdown_connections 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fe0e4037ee0 0x7fe0e403a3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 --2- 172.21.15.125:0/351080979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe10409bea0 0x7fe104129c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 >> 172.21.15.125:0/351080979 conn(0x7fe10409fae0 msgr2=0x7fe10409a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 shutdown_connections 2024-02-20T02:09:44.833 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.832+0000 7fe10b06d640 1 -- 172.21.15.125:0/351080979 wait complete. 2024-02-20T02:09:44.842 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-02-20T02:09:44.842 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-20T02:09:44.948 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.946+0000 7f3b5af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.950+0000 7f3b5af9d640 1 Processor -- start 2024-02-20T02:09:44.949 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.950+0000 7f3b5af9d640 1 -- start start 2024-02-20T02:09:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b5af9d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3b5409bea0 0x7f3b5412cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b5af9d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5409b710 con 0x7f3b5412d2d0 2024-02-20T02:09:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b5af9d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5406eb50 con 0x7f3b5409bea0 2024-02-20T02:09:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b58d39640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3b5409bea0 0x7f3b5412cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b53fff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f3b5412d2d0 0x7f3b5412ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36240/0 (socket says 172.21.15.125:36240) 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b58d39640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f3b5409bea0 0x7f3b5412cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53456/0 (socket says 172.21.15.125:53456) 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b53fff640 1 -- 172.21.15.125:0/1841594267 learned_addr learned my addr 172.21.15.125:0/1841594267 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b58d39640 1 -- 172.21.15.125:0/1841594267 >> v1:172.21.15.125:6789/0 conn(0x7f3b5412d2d0 legacy=0x7f3b5412ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b58d39640 1 -- 172.21.15.125:0/1841594267 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5406e070 con 0x7f3b5409bea0 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.951+0000 7f3b58d39640 1 --2- 172.21.15.125:0/1841594267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b5412cd90 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee4e72430753eb server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b537fe640 1 -- 172.21.15.125:0/1841594267 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b44009ce0 con 0x7f3b5409bea0 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b537fe640 1 -- 172.21.15.125:0/1841594267 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b44009e40 con 0x7f3b5409bea0 2024-02-20T02:09:44.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b537fe640 1 -- 172.21.15.125:0/1841594267 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b4400f470 con 0x7f3b5409bea0 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 -- 172.21.15.125:0/1841594267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 msgr2=0x7f3b5412cd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/1841594267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b5412cd90 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 -- 172.21.15.125:0/1841594267 shutdown_connections 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/1841594267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b5412cd90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 -- 172.21.15.125:0/1841594267 >> 172.21.15.125:0/1841594267 conn(0x7f3b5409fae0 msgr2=0x7f3b5409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 -- 172.21.15.125:0/1841594267 shutdown_connections 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.952+0000 7f3b5af9d640 1 -- 172.21.15.125:0/1841594267 wait complete. 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.953+0000 7f3b5af9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:44.952 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.953+0000 7f3b5af9d640 1 Processor -- start 2024-02-20T02:09:44.953 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.953+0000 7f3b5af9d640 1 -- start start 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.954+0000 7f3b5af9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.954+0000 7f3b5af9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54119ef0 con 0x7f3b5409bea0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.954+0000 7f3b58d39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.954+0000 7f3b58d39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53472/0 (socket says 172.21.15.125:53472) 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.954+0000 7f3b58d39640 1 -- 172.21.15.125:0/2302467967 learned_addr learned my addr 172.21.15.125:0/2302467967 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b58d39640 1 -- 172.21.15.125:0/2302467967 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b54120510 con 0x7f3b5409bea0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b58d39640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b44013e50 con 0x7f3b5409bea0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b44011420 con 0x7f3b5409bea0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b440116c0 con 0x7f3b5409bea0 2024-02-20T02:09:44.954 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5411f730 con 0x7f3b5409bea0 2024-02-20T02:09:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.955+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b54120f00 con 0x7f3b5409bea0 2024-02-20T02:09:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.956+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f3b44011820 con 0x7f3b5409bea0 2024-02-20T02:09:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.956+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5411f3a0 con 0x7f3b5409bea0 2024-02-20T02:09:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.956+0000 7f3b517fa640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 0x7f3b4003a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:44.955 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.956+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3b44018040 con 0x7f3b5409bea0 2024-02-20T02:09:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.956+0000 7f3b53fff640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 0x7f3b4003a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:44.956 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.957+0000 7f3b53fff640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 0x7f3b4003a3f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:44.959 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:44.961+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f3b440188f0 con 0x7f3b5409bea0 2024-02-20T02:09:45.133 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.135+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b54062a30 con 0x7f3b5409bea0 2024-02-20T02:09:45.134 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.135+0000 7f3b517fa640 1 -- 172.21.15.125:0/2302467967 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8741 (crc 0 0 0) 0x7f3b44018710 con 0x7f3b5409bea0 2024-02-20T02:09:45.134 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:45.134 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":14,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:09:44.151306+0000","last_up_change":"2024-02-20T02:09:39.134569+0000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-20T02:09:38.470855+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-02-20T02:09:42.206626+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3754305691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":3754305691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6809","nonce":3754305691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3754305691}]},"public_addr":"172.21.15.125:6802/3754305691","cluster_addr":"172.21.15.125:6801/3754305691","heartbeat_back_addr":"172.21.15.125:6809/3754305691","heartbeat_front_addr":"172.21.15.125:6810/3754305691","state":["exists","up"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3877308034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":3877308034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3877308034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":3877308034}]},"public_addr":"172.21.15.125:6804/3877308034","cluster_addr":"172.21.15.125:6805/3877308034","heartbeat_back_addr":"172.21.15.125:6806/3877308034","heartbeat_front_addr":"172.21.15.125:6803/3877308034","state":["exists","up"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":774159808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":774159808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":774159808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":774159808}]},"public_addr":"172.21.15.204:6811/774159808","cluster_addr":"172.21.15.204:6812/774159808","heartbeat_back_addr":"172.21.15.204:6808/774159808","heartbeat_front_addr":"172.21.15.204:6807/774159808","state":["exists","up"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":409650538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":409650538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":409650538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":409650538}]},"public_addr":"172.21.15.204:6804/409650538","cluster_addr":"172.21.15.204:6803/409650538","heartbeat_back_addr":"172.21.15.204:6810/409650538","heartbeat_front_addr":"172.21.15.204:6809/409650538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:09:45.136 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 msgr2=0x7f3b4003a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.136 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 0x7f3b4003a3f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.136 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 msgr2=0x7f3b541189e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 shutdown_connections 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f3b40037ee0 0x7f3b4003a3f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 --2- 172.21.15.125:0/2302467967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3b5409bea0 0x7f3b541189e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 >> 172.21.15.125:0/2302467967 conn(0x7f3b5409fae0 msgr2=0x7f3b5409cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 shutdown_connections 2024-02-20T02:09:45.137 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.137+0000 7f3b5af9d640 1 -- 172.21.15.125:0/2302467967 wait complete. 2024-02-20T02:09:45.148 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-02-20T02:09:45.148 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-02-20T02:09:45.149 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-02-20T02:09:45.149 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-02-20T02:09:45.265 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.262+0000 7fdbd851d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.266 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.262+0000 7f47f7425640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.266 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.262+0000 7f5159f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.266 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.262+0000 7f73ab99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7f47f7425640 1 Processor -- start 2024-02-20T02:09:45.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7fdbd851d640 1 Processor -- start 2024-02-20T02:09:45.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7fdbd851d640 1 -- start start 2024-02-20T02:09:45.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7f73ab99d640 1 Processor -- start 2024-02-20T02:09:45.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7f5159f9d640 1 Processor -- start 2024-02-20T02:09:45.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7f5159f9d640 1 -- start start 2024-02-20T02:09:45.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.267+0000 7f47f7425640 1 -- start start 2024-02-20T02:09:45.269 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f73ab99d640 1 -- start start 2024-02-20T02:09:45.269 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f7425640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f47f0131c50 0x7f47f0132060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.269 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f5159f9d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f515412d4a0 0x7f5154135270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f7425640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f0131890 con 0x7f47f006ae20 2024-02-20T02:09:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f7425640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f0072db0 con 0x7f47f0131c50 2024-02-20T02:09:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f5159f9d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515409b710 con 0x7f515409bea0 2024-02-20T02:09:45.270 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f5159f9d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515406eb50 con 0x7f515412d4a0 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7fdbd851d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fdbd009e170 0x7fdbd00a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7fdbd851d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00a0260 con 0x7fdbd00a0c60 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7fdbd851d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd009d450 con 0x7fdbd009e170 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f73ab99d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f73a41342a0 0x7f73a41346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f49c0640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f47f006ae20 0x7f47f006b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36254/0 (socket says 172.21.15.125:36254) 2024-02-20T02:09:45.271 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f49c0640 1 -- 172.21.15.125:0/3579360186 learned_addr learned my addr 172.21.15.125:0/3579360186 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f51537fe640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f515409bea0 0x7f515412cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36252/0 (socket says 172.21.15.125:36252) 2024-02-20T02:09:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f73ab99d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a409a810 con 0x7f73a40a17d0 2024-02-20T02:09:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f73ab99d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a4099f20 con 0x7f73a41342a0 2024-02-20T02:09:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f51537fe640 1 -- 172.21.15.125:0/479402868 learned_addr learned my addr 172.21.15.125:0/479402868 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.272 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.268+0000 7f47f51c1640 1 --2- 172.21.15.125:0/3579360186 >> v2:172.21.15.125:3300/0 conn(0x7f47f0131c50 0x7f47f0132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f73a9739640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f73a41342a0 0x7f73a41346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f73a9739640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f73a41342a0 0x7f73a41346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53510/0 (socket says 172.21.15.125:53510) 2024-02-20T02:09:45.273 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f73a9739640 1 -- 172.21.15.125:0/3552420932 learned_addr learned my addr 172.21.15.125:0/3552420932 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fdbd009e170 0x7fdbd00a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fdbd009e170 0x7fdbd00a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53496/0 (socket says 172.21.15.125:53496) 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 -- 172.21.15.125:0/1762600574 learned_addr learned my addr 172.21.15.125:0/1762600574 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f47f51c1640 1 -- 172.21.15.125:0/3579360186 >> v1:172.21.15.125:6789/0 conn(0x7f47f006ae20 legacy=0x7f47f006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f47f51c1640 1 -- 172.21.15.125:0/3579360186 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f009c700 con 0x7f47f0131c50 2024-02-20T02:09:45.274 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 -- 172.21.15.125:0/1762600574 >> v1:172.21.15.125:6789/0 conn(0x7fdbd00a0c60 legacy=0x7fdbd0138510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-20T02:09:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 -- 172.21.15.125:0/1762600574 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd009da90 con 0x7fdbd009e170 2024-02-20T02:09:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f5152ffd640 1 --2- 172.21.15.125:0/479402868 >> v2:172.21.15.125:3300/0 conn(0x7f515412d4a0 0x7f5154135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd62b9640 1 --2- 172.21.15.125:0/1762600574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd00a0720 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b66abaab586e0a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.275 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f73a9739640 1 -- 172.21.15.125:0/3552420932 >> v1:172.21.15.125:6789/0 conn(0x7f73a40a17d0 legacy=0x7f73a40a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-20T02:09:45.276 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f73a9739640 1 -- 172.21.15.125:0/3552420932 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a409bbe0 con 0x7f73a41342a0 2024-02-20T02:09:45.276 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7f47f51c1640 1 --2- 172.21.15.125:0/3579360186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f0131c50 0x7f47f0132060 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bafd6059b75d53 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.276 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.269+0000 7fdbd52b7640 1 -- 172.21.15.125:0/1762600574 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdbcc009d40 con 0x7fdbd009e170 2024-02-20T02:09:45.276 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f73a9739640 1 --2- 172.21.15.125:0/3552420932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a41342a0 0x7f73a41346b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675558770af229a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f5152ffd640 1 -- 172.21.15.125:0/479402868 >> v1:172.21.15.125:6789/0 conn(0x7f515409bea0 legacy=0x7f515412cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f51527fc640 1 -- 172.21.15.125:0/479402868 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209438915 0 0) 0x7f515409b710 con 0x7f515409bea0 2024-02-20T02:09:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f5152ffd640 1 -- 172.21.15.125:0/479402868 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515406e070 con 0x7f515412d4a0 2024-02-20T02:09:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f47e7fff640 1 -- 172.21.15.125:0/3579360186 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47ec009ce0 con 0x7f47f0131c50 2024-02-20T02:09:45.277 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7fdbd52b7640 1 -- 172.21.15.125:0/1762600574 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdbcc00e230 con 0x7fdbd009e170 2024-02-20T02:09:45.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7fdbd52b7640 1 -- 172.21.15.125:0/1762600574 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdbcc009ea0 con 0x7fdbd009e170 2024-02-20T02:09:45.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f47e7fff640 1 -- 172.21.15.125:0/3579360186 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f47ec009e40 con 0x7f47f0131c50 2024-02-20T02:09:45.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f47e7fff640 1 -- 172.21.15.125:0/3579360186 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47ec00e360 con 0x7f47f0131c50 2024-02-20T02:09:45.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f739bfff640 1 -- 172.21.15.125:0/3552420932 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f73a0009d70 con 0x7f73a41342a0 2024-02-20T02:09:45.278 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f5152ffd640 1 --2- 172.21.15.125:0/479402868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515412d4a0 0x7f5154135270 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f90391d2cec661f server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f739bfff640 1 -- 172.21.15.125:0/3552420932 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f73a000e230 con 0x7f73a41342a0 2024-02-20T02:09:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f739bfff640 1 -- 172.21.15.125:0/3552420932 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f73a000e3a0 con 0x7f73a41342a0 2024-02-20T02:09:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f51527fc640 1 -- 172.21.15.125:0/479402868 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5148009ce0 con 0x7f515412d4a0 2024-02-20T02:09:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f51527fc640 1 -- 172.21.15.125:0/479402868 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5148009e40 con 0x7f515412d4a0 2024-02-20T02:09:45.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.270+0000 7f51527fc640 1 -- 172.21.15.125:0/479402868 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f51480104e0 con 0x7f515412d4a0 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 -- 172.21.15.125:0/1762600574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 msgr2=0x7fdbd00a0720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 --2- 172.21.15.125:0/1762600574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd00a0720 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 -- 172.21.15.125:0/479402868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515412d4a0 msgr2=0x7f5154135270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 --2- 172.21.15.125:0/479402868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515412d4a0 0x7f5154135270 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 -- 172.21.15.125:0/479402868 shutdown_connections 2024-02-20T02:09:45.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 --2- 172.21.15.125:0/479402868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515412d4a0 0x7f5154135270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 -- 172.21.15.125:0/479402868 >> 172.21.15.125:0/479402868 conn(0x7f515409fae0 msgr2=0x7f515409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 -- 172.21.15.125:0/1762600574 shutdown_connections 2024-02-20T02:09:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 --2- 172.21.15.125:0/1762600574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd00a0720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 -- 172.21.15.125:0/1762600574 >> 172.21.15.125:0/1762600574 conn(0x7fdbd00a29c0 msgr2=0x7fdbd00a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 -- 172.21.15.125:0/1762600574 shutdown_connections 2024-02-20T02:09:45.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 -- 172.21.15.125:0/3579360186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f0131c50 msgr2=0x7f47f0132060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 --2- 172.21.15.125:0/3579360186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f0131c50 0x7f47f0132060 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 1 -- 172.21.15.125:0/1762600574 wait complete. 2024-02-20T02:09:45.282 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 -- 172.21.15.125:0/479402868 shutdown_connections 2024-02-20T02:09:45.283 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 -- 172.21.15.125:0/3552420932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a41342a0 msgr2=0x7f73a41346b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.283 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 1 -- 172.21.15.125:0/479402868 wait complete. 2024-02-20T02:09:45.283 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 --2- 172.21.15.125:0/3552420932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a41342a0 0x7f73a41346b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.283 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 -- 172.21.15.125:0/3579360186 shutdown_connections 2024-02-20T02:09:45.284 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 -- 172.21.15.125:0/3552420932 shutdown_connections 2024-02-20T02:09:45.284 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 --2- 172.21.15.125:0/3579360186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f0131c50 0x7f47f0132060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.284 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 --2- 172.21.15.125:0/3552420932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a41342a0 0x7f73a41346b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.284 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 -- 172.21.15.125:0/3552420932 >> 172.21.15.125:0/3552420932 conn(0x7f73a406c8a0 msgr2=0x7f73a406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 -- 172.21.15.125:0/3579360186 >> 172.21.15.125:0/3579360186 conn(0x7f47f009f890 msgr2=0x7f47f009fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7fdbd851d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 -- 172.21.15.125:0/3552420932 shutdown_connections 2024-02-20T02:09:45.285 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f5159f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f73ab99d640 1 -- 172.21.15.125:0/3552420932 wait complete. 2024-02-20T02:09:45.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 -- 172.21.15.125:0/3579360186 shutdown_connections 2024-02-20T02:09:45.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.271+0000 7f47f7425640 1 -- 172.21.15.125:0/3579360186 wait complete. 2024-02-20T02:09:45.286 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.272+0000 7f73ab99d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.287 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.272+0000 7f47f7425640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.287 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f47f7425640 1 Processor -- start 2024-02-20T02:09:45.287 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f47f7425640 1 -- start start 2024-02-20T02:09:45.287 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f73ab99d640 1 Processor -- start 2024-02-20T02:09:45.287 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f73ab99d640 1 -- start start 2024-02-20T02:09:45.288 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7fdbd851d640 1 Processor -- start 2024-02-20T02:09:45.288 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f5159f9d640 1 Processor -- start 2024-02-20T02:09:45.288 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7fdbd851d640 1 -- start start 2024-02-20T02:09:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.273+0000 7f5159f9d640 1 -- start start 2024-02-20T02:09:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f7425640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f7425640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0123960 con 0x7f47f006ae20 2024-02-20T02:09:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f51c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.289 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73ab99d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73ab99d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a406db40 con 0x7f73a40a17d0 2024-02-20T02:09:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f51c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53514/0 (socket says 172.21.15.125:53514) 2024-02-20T02:09:45.290 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f51c1640 1 -- 172.21.15.125:0/2851403229 learned_addr learned my addr 172.21.15.125:0/2851403229 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.291 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7fdbd851d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.291 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7fdbd851d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01c5350 con 0x7fdbd009e170 2024-02-20T02:09:45.291 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73a9739640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.291 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73a9739640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53518/0 (socket says 172.21.15.125:53518) 2024-02-20T02:09:45.291 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73a9739640 1 -- 172.21.15.125:0/1828362534 learned_addr learned my addr 172.21.15.125:0/1828362534 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.292 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f5159f9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.292 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f5159f9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5154120f00 con 0x7f515409bea0 2024-02-20T02:09:45.292 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7fdbd62b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.292 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f51537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7fdbd62b9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53528/0 (socket says 172.21.15.125:53528) 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7fdbd62b9640 1 -- 172.21.15.125:0/3946202452 learned_addr learned my addr 172.21.15.125:0/3946202452 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f51537fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53540/0 (socket says 172.21.15.125:53540) 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f51537fe640 1 -- 172.21.15.125:0/5864897 learned_addr learned my addr 172.21.15.125:0/5864897 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f47f51c1640 1 -- 172.21.15.125:0/2851403229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f009c160 con 0x7f47f006ae20 2024-02-20T02:09:45.293 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.274+0000 7f73a9739640 1 -- 172.21.15.125:0/1828362534 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a411ed50 con 0x7f73a40a17d0 2024-02-20T02:09:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbd62b9640 1 -- 172.21.15.125:0/3946202452 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd01ab040 con 0x7fdbd009e170 2024-02-20T02:09:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f51537fe640 1 -- 172.21.15.125:0/5864897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515406bfc0 con 0x7f515409bea0 2024-02-20T02:09:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbd62b9640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.294 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f47f51c1640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f51537fe640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f73a9739640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.295 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f73a0013400 con 0x7f73a40a17d0 2024-02-20T02:09:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdbcc014540 con 0x7fdbd009e170 2024-02-20T02:09:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd01c5580 con 0x7fdbd009e170 2024-02-20T02:09:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdbcc0146a0 con 0x7fdbd009e170 2024-02-20T02:09:45.296 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdbcc014540 con 0x7fdbd009e170 2024-02-20T02:09:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a411f6e0 con 0x7f73a40a17d0 2024-02-20T02:09:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f73a0013560 con 0x7f73a40a17d0 2024-02-20T02:09:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f73a0013400 con 0x7f73a40a17d0 2024-02-20T02:09:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41311e0 con 0x7f73a40a17d0 2024-02-20T02:09:45.297 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd01ab6a0 con 0x7fdbd009e170 2024-02-20T02:09:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47ec014620 con 0x7f47f006ae20 2024-02-20T02:09:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.275+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f0122d00 con 0x7f47f006ae20 2024-02-20T02:09:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f0121400 con 0x7f47f006ae20 2024-02-20T02:09:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f47ec014780 con 0x7f47f006ae20 2024-02-20T02:09:45.298 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f73a0013770 con 0x7f73a40a17d0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f7399ffb640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 0x7f7388042bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f73a0018040 con 0x7f73a40a17d0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5144004af0 con 0x7f515409bea0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5144004c50 con 0x7f515409bea0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5144007120 con 0x7f515409bea0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f5159f9d640 1 -- 172.21.15.125:0/5864897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515411e9a0 con 0x7f515409bea0 2024-02-20T02:09:45.299 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73a41211a0 con 0x7f73a40a17d0 2024-02-20T02:09:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f5159f9d640 1 -- 172.21.15.125:0/5864897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515411f860 con 0x7f515409bea0 2024-02-20T02:09:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f7399ffb640 1 --2- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 0x7f7388047b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 --> v2:172.21.15.125:6802/3754305691 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7388048200 con 0x7f73880456e0 2024-02-20T02:09:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f73a8f38640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 0x7f7388042bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.300 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f73a0018b20 con 0x7f73a40a17d0 2024-02-20T02:09:45.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.276+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbd01acd60 con 0x7fdbd009e170 2024-02-20T02:09:45.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47d0000f80 con 0x7f47f006ae20 2024-02-20T02:09:45.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47ec014a20 con 0x7f47f006ae20 2024-02-20T02:09:45.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fdbcc014860 con 0x7fdbd009e170 2024-02-20T02:09:45.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7fdbc6ffd640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 0x7fdbb403a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.277+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fdbcc019040 con 0x7fdbd009e170 2024-02-20T02:09:45.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbc6ffd640 1 --2- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 0x7fdbb403f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f73a9f3a640 1 --2- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 0x7f7388047b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:09:45.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 --> v2:172.21.15.204:6811/774159808 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbb403fa30 con 0x7fdbb403cf10 2024-02-20T02:09:45.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7fdbcc014540 con 0x7fdbd009e170 2024-02-20T02:09:45.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbd5ab8640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 0x7fdbb403a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f515411f220 con 0x7f515409bea0 2024-02-20T02:09:45.303 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f73a8f38640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 0x7f7388042bf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.304 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f5144007320 con 0x7f515409bea0 2024-02-20T02:09:45.304 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f47ec011380 con 0x7f47f006ae20 2024-02-20T02:09:45.304 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 0x7f514003a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.304 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f51440474a0 con 0x7f515409bea0 2024-02-20T02:09:45.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 0x7f47d403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 --2- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 0x7f514003f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 --> v2:172.21.15.204:6804/409650538 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f514003fa00 con 0x7f514003cee0 2024-02-20T02:09:45.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f47ec019040 con 0x7f47f006ae20 2024-02-20T02:09:45.306 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f51440477e0 con 0x7f515409bea0 2024-02-20T02:09:45.306 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbd5ab8640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 0x7fdbb403a420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.306 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 --2- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 0x7f47d403f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 --> v2:172.21.15.125:6804/3877308034 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47d403fa00 con 0x7f47d403cee0 2024-02-20T02:09:45.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (crc 0 0 0) 0x7f47ec04b0b0 con 0x7f47f006ae20 2024-02-20T02:09:45.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7fdbd6aba640 1 --2- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 0x7fdbb403f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:09:45.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.278+0000 7f5152ffd640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 0x7f514003a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.307 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.279+0000 7f5153fff640 1 --2- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 0x7f514003f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:09:45.308 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.279+0000 7f47f49c0640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 0x7f47d403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.308 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.279+0000 7f47f59c2640 1 --2- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 0x7f47d403f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:09:45.308 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.279+0000 7f47f49c0640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 0x7f47d403a3f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.308 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.279+0000 7f5152ffd640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 0x7f514003a3f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.309 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.280+0000 7f73a9f3a640 1 --2- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 0x7f7388047b70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.309 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.280+0000 7f47f59c2640 1 --2- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 0x7f47d403f370 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.309 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.281+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== osd.0 v2:172.21.15.125:6802/3754305691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f7388048200 con 0x7f73880456e0 2024-02-20T02:09:45.309 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.281+0000 7fdbd6aba640 1 --2- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 0x7fdbb403f3a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.310 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.281+0000 7f5153fff640 1 --2- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 0x7f514003f370 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.310 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.281+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== osd.1 v2:172.21.15.125:6804/3877308034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f47d403fa00 con 0x7f47d403cee0 2024-02-20T02:09:45.310 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.282+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== osd.2 v2:172.21.15.204:6811/774159808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fdbb403fa30 con 0x7fdbb403cf10 2024-02-20T02:09:45.311 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.282+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== osd.3 v2:172.21.15.204:6804/409650538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f514003fa00 con 0x7f514003cee0 2024-02-20T02:09:45.311 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.284+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 --> v2:172.21.15.125:6802/3754305691 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f73a4074a70 con 0x7f73880456e0 2024-02-20T02:09:45.311 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.285+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 --> v2:172.21.15.204:6804/409650538 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f515412caf0 con 0x7f514003cee0 2024-02-20T02:09:45.311 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.285+0000 7f7399ffb640 1 -- 172.21.15.125:0/1828362534 <== osd.0 v2:172.21.15.125:6802/3754305691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f73a4074a70 con 0x7f73880456e0 2024-02-20T02:09:45.312 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 msgr2=0x7f7388047b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.312 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 0x7f7388047b70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.312 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f513bfff640 1 -- 172.21.15.125:0/5864897 <== osd.3 v2:172.21.15.204:6804/409650538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f514c00a820 con 0x7f514003cee0 2024-02-20T02:09:45.312 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 msgr2=0x7f7388042bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.313 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 0x7f7388042bf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.313 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 msgr2=0x7f73a411d260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.313 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.313 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 shutdown_connections 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> v2:172.21.15.125:6802/3754305691 conn(0x7f73880456e0 0x7f7388047b70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f73880406e0 0x7f7388042bf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 --2- 172.21.15.125:0/1828362534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73a40a17d0 0x7f73a411d260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 >> 172.21.15.125:0/1828362534 conn(0x7f73a406c8a0 msgr2=0x7f73a40a3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.286+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 shutdown_connections 2024-02-20T02:09:45.314 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f73ab99d640 1 -- 172.21.15.125:0/1828362534 wait complete. 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 msgr2=0x7f514003f370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 0x7f514003f370 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 msgr2=0x7f514003a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 0x7f514003a3f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 msgr2=0x7f515411d6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 shutdown_connections 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> v2:172.21.15.204:6804/409650538 conn(0x7f514003cee0 0x7f514003f370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5140037ee0 0x7f514003a3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.315 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 --2- 172.21.15.125:0/5864897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f515409bea0 0x7f515411d6e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.316 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 >> 172.21.15.125:0/5864897 conn(0x7f515409fae0 msgr2=0x7f515412fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.316 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 shutdown_connections 2024-02-20T02:09:45.316 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.287+0000 7f5139ffb640 1 -- 172.21.15.125:0/5864897 wait complete. 2024-02-20T02:09:45.316 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.288+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 --> v2:172.21.15.125:6804/3877308034 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47d0002ba0 con 0x7f47d403cee0 2024-02-20T02:09:45.316 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.288+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 --> v2:172.21.15.204:6811/774159808 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbd0099a70 con 0x7fdbb403cf10 2024-02-20T02:09:45.317 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.289+0000 7f47e5ffb640 1 -- 172.21.15.125:0/2851403229 <== osd.1 v2:172.21.15.125:6804/3877308034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47d0002ba0 con 0x7f47d403cee0 2024-02-20T02:09:45.317 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.289+0000 7fdbc6ffd640 1 -- 172.21.15.125:0/3946202452 <== osd.2 v2:172.21.15.204:6811/774159808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbd0099a70 con 0x7fdbb403cf10 2024-02-20T02:09:45.317 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 msgr2=0x7f47d403f370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.317 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 0x7f47d403f370 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 msgr2=0x7fdbb403f3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 0x7fdbb403f3a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 msgr2=0x7fdbb403a420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 0x7fdbb403a420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 msgr2=0x7fdbd01c5890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.318 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 msgr2=0x7f47d403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 0x7f47d403a3f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 shutdown_connections 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> v2:172.21.15.204:6811/774159808 conn(0x7fdbb403cf10 0x7fdbb403f3a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fdbb4037f10 0x7fdbb403a420 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 --2- 172.21.15.125:0/3946202452 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdbd009e170 0x7fdbd01c5890 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.319 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 >> 172.21.15.125:0/3946202452 conn(0x7fdbd00a29c0 msgr2=0x7fdbd009b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.320 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 msgr2=0x7f47f011f880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.320 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.320 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 shutdown_connections 2024-02-20T02:09:45.320 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7fdbd851d640 1 -- 172.21.15.125:0/3946202452 wait complete. 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 shutdown_connections 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> v2:172.21.15.125:6804/3877308034 conn(0x7f47d403cee0 0x7f47d403f370 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f47d4037ee0 0x7f47d403a3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 --2- 172.21.15.125:0/2851403229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f47f006ae20 0x7f47f011f880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 >> 172.21.15.125:0/2851403229 conn(0x7f47f009f890 msgr2=0x7f47f012e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 shutdown_connections 2024-02-20T02:09:45.321 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.290+0000 7f47f7425640 1 -- 172.21.15.125:0/2851403229 wait complete. 2024-02-20T02:09:45.322 INFO:teuthology.orchestra.run.smithi125.stdout:38654705671 2024-02-20T02:09:45.322 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-20T02:09:45.323 INFO:teuthology.orchestra.run.smithi125.stdout:42949672966 2024-02-20T02:09:45.323 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-20T02:09:45.324 INFO:teuthology.orchestra.run.smithi125.stdout:42949672966 2024-02-20T02:09:45.324 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-20T02:09:45.325 INFO:teuthology.orchestra.run.smithi125.stdout:38654705671 2024-02-20T02:09:45.325 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-20T02:09:45.480 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.477+0000 7fd9d759d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.480 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.477+0000 7fb29c02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.480 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.477+0000 7fd53599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.481 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.477+0000 7f5d2bedd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.481 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.481+0000 7fd9d759d640 1 Processor -- start 2024-02-20T02:09:45.481 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.481+0000 7fb29c02d640 1 Processor -- start 2024-02-20T02:09:45.482 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd53599d640 1 Processor -- start 2024-02-20T02:09:45.482 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd53599d640 1 -- start start 2024-02-20T02:09:45.482 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd9d759d640 1 -- start start 2024-02-20T02:09:45.482 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fb29c02d640 1 -- start start 2024-02-20T02:09:45.483 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7f5d2bedd640 1 Processor -- start 2024-02-20T02:09:45.483 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7f5d2bedd640 1 -- start start 2024-02-20T02:09:45.483 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd9d759d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fd9d0131c50 0x7fd9d0132060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.484 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd53599d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fd5301342a0 0x7fd5301346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.484 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd9d759d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d0131890 con 0x7fd9d006ae20 2024-02-20T02:09:45.484 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.482+0000 7fd9d759d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d0072db0 con 0x7fd9d0131c50 2024-02-20T02:09:45.484 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb29c02d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fb294130a10 0x7fb294135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd53599d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53009a810 con 0x7fd5300a17d0 2024-02-20T02:09:45.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd53599d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd530099f20 con 0x7fd5301342a0 2024-02-20T02:09:45.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9d4b38640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fd9d006ae20 0x7fd9d006b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36282/0 (socket says 172.21.15.125:36282) 2024-02-20T02:09:45.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb29c02d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29409ba10 con 0x7fb29409be00 2024-02-20T02:09:45.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9d4b38640 1 -- 172.21.15.125:0/3075045234 learned_addr learned my addr 172.21.15.125:0/3075045234 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb29c02d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb294130e20 con 0x7fb294130a10 2024-02-20T02:09:45.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52e7fc640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fd5300a17d0 0x7fd5300a1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36292/0 (socket says 172.21.15.125:36292) 2024-02-20T02:09:45.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52e7fc640 1 -- 172.21.15.125:0/2250854056 learned_addr learned my addr 172.21.15.125:0/2250854056 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52effd640 1 --2- 172.21.15.125:0/2250854056 >> v2:172.21.15.125:3300/0 conn(0x7fd5301342a0 0x7fd5301346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9d5339640 1 --2- 172.21.15.125:0/3075045234 >> v2:172.21.15.125:3300/0 conn(0x7fd9d0131c50 0x7fd9d0132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb299dc9640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fb29409be00 0x7fb294130300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36306/0 (socket says 172.21.15.125:36306) 2024-02-20T02:09:45.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb299dc9640 1 -- 172.21.15.125:0/3204454195 learned_addr learned my addr 172.21.15.125:0/3204454195 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb2995c8640 1 --2- 172.21.15.125:0/3204454195 >> v2:172.21.15.125:3300/0 conn(0x7fb294130a10 0x7fb294135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52dffb640 1 -- 172.21.15.125:0/2250854056 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483405572 0 0) 0x7fd53009a810 con 0x7fd5300a17d0 2024-02-20T02:09:45.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52dffb640 1 -- 172.21.15.125:0/2250854056 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5180034f0 con 0x7fd5300a17d0 2024-02-20T02:09:45.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7f5d2bedd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5d2409bec0 0x7f5d2412d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9c7fff640 1 -- 172.21.15.125:0/3075045234 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049392544 0 0) 0x7fd9d0131890 con 0x7fd9d006ae20 2024-02-20T02:09:45.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7f5d2bedd640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2409b8b0 con 0x7f5d2412dca0 2024-02-20T02:09:45.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7f5d2bedd640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2409cda0 con 0x7f5d2409bec0 2024-02-20T02:09:45.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9c7fff640 1 -- 172.21.15.125:0/3075045234 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc0034f0 con 0x7fd9d006ae20 2024-02-20T02:09:45.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb2995c8640 1 -- 172.21.15.125:0/3204454195 >> v1:172.21.15.125:6789/0 conn(0x7fb29409be00 legacy=0x7fb294130300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fb2995c8640 1 -- 172.21.15.125:0/3204454195 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29412fff0 con 0x7fb294130a10 2024-02-20T02:09:45.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52effd640 1 -- 172.21.15.125:0/2250854056 >> v1:172.21.15.125:6789/0 conn(0x7fd5300a17d0 legacy=0x7fd5300a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd52effd640 1 -- 172.21.15.125:0/2250854056 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53009bbe0 con 0x7fd5301342a0 2024-02-20T02:09:45.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7f5d29c79640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5d2409bec0 0x7f5d2412d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d29478640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f5d2412dca0 0x7f5d241307a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36322/0 (socket says 172.21.15.125:36322) 2024-02-20T02:09:45.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d29478640 1 -- 172.21.15.125:0/3135702828 learned_addr learned my addr 172.21.15.125:0/3135702828 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fb2995c8640 1 --2- 172.21.15.125:0/3204454195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294130a10 0x7fb294135220 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81ee9b9ca8d2bf1 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd52effd640 1 --2- 172.21.15.125:0/2250854056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5301342a0 0x7fd5301346b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22814a21224ef73b server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9d5339640 1 -- 172.21.15.125:0/3075045234 >> v1:172.21.15.125:6789/0 conn(0x7fd9d006ae20 legacy=0x7fd9d006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.483+0000 7fd9d5339640 1 -- 172.21.15.125:0/3075045234 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d009c700 con 0x7fd9d0131c50 2024-02-20T02:09:45.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd9d5339640 1 --2- 172.21.15.125:0/3075045234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d0131c50 0x7fd9d0132060 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae97de96aed01b48 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fb298dc7640 1 -- 172.21.15.125:0/3204454195 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb284009ce0 con 0x7fb294130a10 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd52dffb640 1 -- 172.21.15.125:0/2250854056 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd520009cd0 con 0x7fd5301342a0 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd9c7fff640 1 -- 172.21.15.125:0/3075045234 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9cc009ce0 con 0x7fd9d0131c50 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d29c79640 1 -- 172.21.15.125:0/3135702828 >> v1:172.21.15.125:6789/0 conn(0x7f5d2412dca0 legacy=0x7f5d241307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d29c79640 1 -- 172.21.15.125:0/3135702828 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2409b5a0 con 0x7f5d2409bec0 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd52dffb640 1 -- 172.21.15.125:0/2250854056 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd520009e30 con 0x7fd5301342a0 2024-02-20T02:09:45.493 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd52dffb640 1 -- 172.21.15.125:0/2250854056 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd52000e360 con 0x7fd5301342a0 2024-02-20T02:09:45.494 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fb298dc7640 1 -- 172.21.15.125:0/3204454195 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb284009e40 con 0x7fb294130a10 2024-02-20T02:09:45.494 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fb298dc7640 1 -- 172.21.15.125:0/3204454195 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb28400e360 con 0x7fb294130a10 2024-02-20T02:09:45.494 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d29c79640 1 --2- 172.21.15.125:0/3135702828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d2412d760 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f409404fc6995395 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.494 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd9c7fff640 1 -- 172.21.15.125:0/3075045234 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd9cc009e40 con 0x7fd9d0131c50 2024-02-20T02:09:45.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7fd9c7fff640 1 -- 172.21.15.125:0/3075045234 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9cc00e360 con 0x7fd9d0131c50 2024-02-20T02:09:45.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d28c77640 1 -- 172.21.15.125:0/3135702828 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d20009ce0 con 0x7f5d2409bec0 2024-02-20T02:09:45.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d28c77640 1 -- 172.21.15.125:0/3135702828 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d20009e40 con 0x7f5d2409bec0 2024-02-20T02:09:45.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.484+0000 7f5d28c77640 1 -- 172.21.15.125:0/3135702828 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d2000e360 con 0x7f5d2409bec0 2024-02-20T02:09:45.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 -- 172.21.15.125:0/2250854056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5301342a0 msgr2=0x7fd5301346b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.496 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 -- 172.21.15.125:0/3204454195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294130a10 msgr2=0x7fb294135220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.496 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 --2- 172.21.15.125:0/2250854056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5301342a0 0x7fd5301346b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.496 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 -- 172.21.15.125:0/3075045234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d0131c50 msgr2=0x7fd9d0132060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.496 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 --2- 172.21.15.125:0/3204454195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294130a10 0x7fb294135220 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.497 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3075045234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d0131c50 0x7fd9d0132060 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.497 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 -- 172.21.15.125:0/2250854056 shutdown_connections 2024-02-20T02:09:45.497 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 --2- 172.21.15.125:0/2250854056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5301342a0 0x7fd5301346b0 crc :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.497 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 -- 172.21.15.125:0/3204454195 shutdown_connections 2024-02-20T02:09:45.498 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 -- 172.21.15.125:0/2250854056 >> 172.21.15.125:0/2250854056 conn(0x7fd53006c8a0 msgr2=0x7fd53006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.498 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 --2- 172.21.15.125:0/3204454195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb294130a10 0x7fb294135220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.498 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 -- 172.21.15.125:0/3204454195 >> 172.21.15.125:0/3204454195 conn(0x7fb29409fae0 msgr2=0x7fb29409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.498 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 -- 172.21.15.125:0/3075045234 shutdown_connections 2024-02-20T02:09:45.499 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3075045234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d0131c50 0x7fd9d0132060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.499 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 -- 172.21.15.125:0/3075045234 >> 172.21.15.125:0/3075045234 conn(0x7fd9d009f890 msgr2=0x7fd9d009fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.499 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 -- 172.21.15.125:0/2250854056 shutdown_connections 2024-02-20T02:09:45.499 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 -- 172.21.15.125:0/3135702828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 msgr2=0x7f5d2412d760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.499 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 1 -- 172.21.15.125:0/2250854056 wait complete. 2024-02-20T02:09:45.500 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/3135702828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d2412d760 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.500 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 -- 172.21.15.125:0/3135702828 shutdown_connections 2024-02-20T02:09:45.500 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 -- 172.21.15.125:0/3204454195 shutdown_connections 2024-02-20T02:09:45.500 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 -- 172.21.15.125:0/3075045234 shutdown_connections 2024-02-20T02:09:45.500 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd9d759d640 1 -- 172.21.15.125:0/3075045234 wait complete. 2024-02-20T02:09:45.501 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 1 -- 172.21.15.125:0/3204454195 wait complete. 2024-02-20T02:09:45.501 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/3135702828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d2412d760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.501 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 -- 172.21.15.125:0/3135702828 >> 172.21.15.125:0/3135702828 conn(0x7f5d240a0a80 msgr2=0x7f5d240a0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.502 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 -- 172.21.15.125:0/3135702828 shutdown_connections 2024-02-20T02:09:45.502 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7f5d2bedd640 1 -- 172.21.15.125:0/3135702828 wait complete. 2024-02-20T02:09:45.502 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fb29c02d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.502 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.486+0000 7fd9d759d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.502 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.486+0000 7f5d2bedd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.503 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.485+0000 7fd53599d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.503 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7f5d2bedd640 1 Processor -- start 2024-02-20T02:09:45.503 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fd53599d640 1 Processor -- start 2024-02-20T02:09:45.504 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7f5d2bedd640 1 -- start start 2024-02-20T02:09:45.504 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fb29c02d640 1 Processor -- start 2024-02-20T02:09:45.504 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fb29c02d640 1 -- start start 2024-02-20T02:09:45.504 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fd9d759d640 1 Processor -- start 2024-02-20T02:09:45.505 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fd53599d640 1 -- start start 2024-02-20T02:09:45.505 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7fd9d759d640 1 -- start start 2024-02-20T02:09:45.505 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.487+0000 7f5d2bedd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.505 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d2bedd640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24136ab0 con 0x7f5d2409bec0 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d29c79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d29c79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53594/0 (socket says 172.21.15.125:53594) 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d29c79640 1 -- 172.21.15.125:0/268832589 learned_addr learned my addr 172.21.15.125:0/268832589 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb29c02d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb29c02d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29411a280 con 0x7fb29409be00 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb299dc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.506 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb299dc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53608/0 (socket says 172.21.15.125:53608) 2024-02-20T02:09:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb299dc9640 1 -- 172.21.15.125:0/965894219 learned_addr learned my addr 172.21.15.125:0/965894219 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d29c79640 1 -- 172.21.15.125:0/268832589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24136450 con 0x7f5d2409bec0 2024-02-20T02:09:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fd53599d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fd53599d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53011f700 con 0x7fd5300a17d0 2024-02-20T02:09:45.507 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb299dc9640 1 -- 172.21.15.125:0/965894219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2941c4530 con 0x7fb29409be00 2024-02-20T02:09:45.508 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fd52effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.508 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d29c79640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.508 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fb299dc9640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.509 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fd52effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53618/0 (socket says 172.21.15.125:53618) 2024-02-20T02:09:45.509 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7fd52effd640 1 -- 172.21.15.125:0/1378306861 learned_addr learned my addr 172.21.15.125:0/1378306861 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.509 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.488+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d20014620 con 0x7f5d2409bec0 2024-02-20T02:09:45.509 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd9d759d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d20014780 con 0x7f5d2409bec0 2024-02-20T02:09:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d20014a20 con 0x7f5d2409bec0 2024-02-20T02:09:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d24136ce0 con 0x7f5d2409bec0 2024-02-20T02:09:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2900059d0 con 0x7fb29409be00 2024-02-20T02:09:45.510 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb290005b30 con 0x7fb29409be00 2024-02-20T02:09:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd9d759d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d011e510 con 0x7fd9d006ae20 2024-02-20T02:09:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd52effd640 1 -- 172.21.15.125:0/1378306861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53011fc70 con 0x7fd5300a17d0 2024-02-20T02:09:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d24135b70 con 0x7f5d2409bec0 2024-02-20T02:09:45.511 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd52effd640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd9d5339640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd9d5339640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53622/0 (socket says 172.21.15.125:53622) 2024-02-20T02:09:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd9d5339640 1 -- 172.21.15.125:0/3294358846 learned_addr learned my addr 172.21.15.125:0/3294358846 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.512 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd520015420 con 0x7fd5300a17d0 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb290005dd0 con 0x7fb29409be00 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd520015580 con 0x7fd5300a17d0 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd520015820 con 0x7fd5300a17d0 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53011fea0 con 0x7fd5300a17d0 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29411b490 con 0x7fb29409be00 2024-02-20T02:09:45.513 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29411cd80 con 0x7fb29409be00 2024-02-20T02:09:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.489+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5301211b0 con 0x7fd5300a17d0 2024-02-20T02:09:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7fd9d5339640 1 -- 172.21.15.125:0/3294358846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d009c160 con 0x7fd9d006ae20 2024-02-20T02:09:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7fd9d5339640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.514 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d24119670 con 0x7f5d2409bec0 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f5d20014b80 con 0x7f5d2409bec0 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7f5d1a7fc640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 0x7f5d0803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.490+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5d20019040 con 0x7f5d2409bec0 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530062a30 con 0x7fd5300a17d0 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fd520015a10 con 0x7fd5300a17d0 2024-02-20T02:09:45.515 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9cc014540 con 0x7fd9d006ae20 2024-02-20T02:09:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d011c1a0 con 0x7fd9d006ae20 2024-02-20T02:09:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7f5d29478640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 0x7f5d0803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fb2900070b0 con 0x7fb29409be00 2024-02-20T02:09:45.516 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29411b620 con 0x7fb29409be00 2024-02-20T02:09:45.517 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d011ca30 con 0x7fd9d006ae20 2024-02-20T02:09:45.517 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd517fff640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 0x7fd51c03a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.517 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd520048c60 con 0x7fd5300a17d0 2024-02-20T02:09:45.517 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fb28a7fc640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 0x7fb27803e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.517 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fd52e7fc640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 0x7fd51c03a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.518 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb290047410 con 0x7fb29409be00 2024-02-20T02:09:45.518 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7fb2995c8640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 0x7fb27803e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.518 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.491+0000 7f5d29478640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 0x7f5d0803a3f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.518 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.492+0000 7fd52e7fc640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 0x7fd51c03a3e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.519 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.492+0000 7fb2995c8640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 0x7fb27803e800 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.519 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.492+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d011d3e0 con 0x7fd9d006ae20 2024-02-20T02:09:45.519 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.496+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd9cc0146a0 con 0x7fd9d006ae20 2024-02-20T02:09:45.519 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.496+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9cc014940 con 0x7fd9d006ae20 2024-02-20T02:09:45.519 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.496+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f5d20018c00 con 0x7f5d2409bec0 2024-02-20T02:09:45.520 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.501+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7fb29000d330 con 0x7fb29409be00 2024-02-20T02:09:45.520 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.510+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7fd52000f2a0 con 0x7fd5300a17d0 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.510+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fd9cc014aa0 con 0x7fd9d006ae20 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.511+0000 7fd9c5ffb640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 0x7fd9b403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.511+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd9cc019040 con 0x7fd9d006ae20 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.511+0000 7fd9d4b38640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 0x7fd9b403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.512+0000 7fd9d4b38640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 0x7fd9b403a3f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.521 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.519+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7fd9cc019ce0 con 0x7fd9d006ae20 2024-02-20T02:09:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.691+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd5300671b0 con 0x7fd5300a17d0 2024-02-20T02:09:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.691+0000 7fd517fff640 1 -- 172.21.15.125:0/1378306861 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fd520004450 con 0x7fd5300a17d0 2024-02-20T02:09:45.690 INFO:teuthology.orchestra.run.smithi125.stdout:38654705671 2024-02-20T02:09:45.690 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.692+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb2941c4970 con 0x7fb29409be00 2024-02-20T02:09:45.691 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.692+0000 7fb28a7fc640 1 -- 172.21.15.125:0/965894219 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fb29000d150 con 0x7fb29409be00 2024-02-20T02:09:45.691 INFO:teuthology.orchestra.run.smithi125.stdout:42949672966 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 msgr2=0x7fd51c03a3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 0x7fd51c03a3e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 msgr2=0x7fd5301c90c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 shutdown_connections 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd51c037ed0 0x7fd51c03a3e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.692 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 --2- 172.21.15.125:0/1378306861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5300a17d0 0x7fd5301c90c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 >> 172.21.15.125:0/1378306861 conn(0x7fd53006c8a0 msgr2=0x7fd5300a3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 shutdown_connections 2024-02-20T02:09:45.693 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.693+0000 7fd53599d640 1 -- 172.21.15.125:0/1378306861 wait complete. 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.694+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 msgr2=0x7fb27803e800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.694+0000 7fb29c02d640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 0x7fb27803e800 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.694+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 msgr2=0x7fb2941c4b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.694+0000 7fb29c02d640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 shutdown_connections 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fb27803c2f0 0x7fb27803e800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 --2- 172.21.15.125:0/965894219 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb29409be00 0x7fb2941c4b70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 >> 172.21.15.125:0/965894219 conn(0x7fb29409fae0 msgr2=0x7fb29412c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 shutdown_connections 2024-02-20T02:09:45.694 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7fb29c02d640 1 -- 172.21.15.125:0/965894219 wait complete. 2024-02-20T02:09:45.695 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5d24062a30 con 0x7f5d2409bec0 2024-02-20T02:09:45.695 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.695+0000 7f5d1a7fc640 1 -- 172.21.15.125:0/268832589 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f5d2000e230 con 0x7f5d2409bec0 2024-02-20T02:09:45.695 INFO:teuthology.orchestra.run.smithi125.stdout:42949672966 2024-02-20T02:09:45.697 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.698+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 msgr2=0x7f5d0803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.698+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 0x7f5d0803a3f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.698+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 msgr2=0x7f5d24134080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.698+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 shutdown_connections 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5d08037ee0 0x7f5d0803a3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 --2- 172.21.15.125:0/268832589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5d2409bec0 0x7f5d24134080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 >> 172.21.15.125:0/268832589 conn(0x7f5d240a0a80 msgr2=0x7f5d2412ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 shutdown_connections 2024-02-20T02:09:45.698 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.699+0000 7f5d2bedd640 1 -- 172.21.15.125:0/268832589 wait complete. 2024-02-20T02:09:45.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.1 2024-02-20T02:09:45.706 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:45.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.2 2024-02-20T02:09:45.707 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:45.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.3 2024-02-20T02:09:45.712 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:45.718 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.720+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd9d0062a30 con 0x7fd9d006ae20 2024-02-20T02:09:45.718 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.720+0000 7fd9c5ffb640 1 -- 172.21.15.125:0/3294358846 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fd9cc001d40 con 0x7fd9d006ae20 2024-02-20T02:09:45.719 INFO:teuthology.orchestra.run.smithi125.stdout:38654705671 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 msgr2=0x7fd9b403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 0x7fd9b403a3f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 msgr2=0x7fd9d011aea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 shutdown_connections 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd9b4037ee0 0x7fd9b403a3f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 --2- 172.21.15.125:0/3294358846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9d006ae20 0x7fd9d011aea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 >> 172.21.15.125:0/3294358846 conn(0x7fd9d009f890 msgr2=0x7fd9d00a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 shutdown_connections 2024-02-20T02:09:45.721 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.722+0000 7fd9d759d640 1 -- 172.21.15.125:0/3294358846 wait complete. 2024-02-20T02:09:45.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-02-20T02:09:45.731 DEBUG:teuthology.parallel:result is None 2024-02-20T02:09:45.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-20T02:09:45.732 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:09:45.878 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.876+0000 7f9633edd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.880+0000 7f9633edd640 1 Processor -- start 2024-02-20T02:09:45.879 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9633edd640 1 -- start start 2024-02-20T02:09:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9633edd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f962c09e170 0x7f962c0a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9633edd640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0a0260 con 0x7f962c0a0c60 2024-02-20T02:09:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9633edd640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c09d450 con 0x7f962c09e170 2024-02-20T02:09:45.881 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9631c79640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f962c09e170 0x7f962c0a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9631478640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f962c0a0c60 0x7f962c138510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36332/0 (socket says 172.21.15.125:36332) 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.881+0000 7f9631478640 1 -- 172.21.15.125:0/1535947928 learned_addr learned my addr 172.21.15.125:0/1535947928 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9631c79640 1 -- 172.21.15.125:0/1535947928 >> v1:172.21.15.125:6789/0 conn(0x7f962c0a0c60 legacy=0x7f962c138510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9631c79640 1 -- 172.21.15.125:0/1535947928 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c09da90 con 0x7f962c09e170 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9630c77640 1 -- 172.21.15.125:0/1535947928 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677254065 0 0) 0x7f962c0a0260 con 0x7f962c0a0c60 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9631c79640 1 --2- 172.21.15.125:0/1535947928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c0a0720 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b91e55660895580 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9630c77640 1 -- 172.21.15.125:0/1535947928 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9624009ce0 con 0x7f962c09e170 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9630c77640 1 -- 172.21.15.125:0/1535947928 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9624009e40 con 0x7f962c09e170 2024-02-20T02:09:45.882 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.882+0000 7f9630c77640 1 -- 172.21.15.125:0/1535947928 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f962400e360 con 0x7f962c09e170 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 -- 172.21.15.125:0/1535947928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 msgr2=0x7f962c0a0720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 --2- 172.21.15.125:0/1535947928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c0a0720 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 -- 172.21.15.125:0/1535947928 shutdown_connections 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 --2- 172.21.15.125:0/1535947928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c0a0720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 -- 172.21.15.125:0/1535947928 >> 172.21.15.125:0/1535947928 conn(0x7f962c0a29c0 msgr2=0x7f962c0a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:45.884 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 -- 172.21.15.125:0/1535947928 shutdown_connections 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 1 -- 172.21.15.125:0/1535947928 wait complete. 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.883+0000 7f9633edd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.884+0000 7f9633edd640 1 Processor -- start 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.884+0000 7f9633edd640 1 -- start start 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9633edd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9633edd640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1c4ff0 con 0x7f962c09e170 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9631c79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9631c79640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53640/0 (socket says 172.21.15.125:53640) 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9631c79640 1 -- 172.21.15.125:0/3173125636 learned_addr learned my addr 172.21.15.125:0/3173125636 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9631c79640 1 -- 172.21.15.125:0/3173125636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c1aace0 con 0x7f962c09e170 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9631c79640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9624014540 con 0x7f962c09e170 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f96240146a0 con 0x7f962c09e170 2024-02-20T02:09:45.885 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9624014940 con 0x7f962c09e170 2024-02-20T02:09:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c1c5220 con 0x7f962c09e170 2024-02-20T02:09:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.885+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c1ab340 con 0x7f962c09e170 2024-02-20T02:09:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.886+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f9624014af0 con 0x7f962c09e170 2024-02-20T02:09:45.886 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.886+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c062a30 con 0x7f962c09e170 2024-02-20T02:09:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.886+0000 7f96227fc640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 0x7f961003a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.887+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9624019040 con 0x7f962c09e170 2024-02-20T02:09:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.887+0000 7f9631478640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 0x7f961003a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:45.887 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.887+0000 7f9631478640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 0x7f961003a3f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:45.890 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:45.891+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f9624018190 con 0x7f962c09e170 2024-02-20T02:09:46.057 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.058+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f962c1aca00 con 0x7f9610037ee0 2024-02-20T02:09:46.058 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.060+0000 7f96227fc640 1 -- 172.21.15.125:0/3173125636 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29957 (crc 0 0 0) 0x7f962c1aca00 con 0x7f9610037ee0 2024-02-20T02:09:46.059 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:46.059 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:09:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 msgr2=0x7f961003a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.060 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 0x7f961003a3f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 msgr2=0x7f962c1c5530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 shutdown_connections 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9610037ee0 0x7f961003a3f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 --2- 172.21.15.125:0/3173125636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f962c09e170 0x7f962c1c5530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 >> 172.21.15.125:0/3173125636 conn(0x7f962c0a29c0 msgr2=0x7f962c09ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 shutdown_connections 2024-02-20T02:09:46.061 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.061+0000 7f9633edd640 1 -- 172.21.15.125:0/3173125636 wait complete. 2024-02-20T02:09:46.071 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-02-20T02:09:45.290927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109748,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320652,"statfs":{"total":429496729600,"available":429384347648,"internally_reserved":0,"allocated":1662976,"data_stored":1300128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.206767+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:09:44.206767+0000","last_peered":"2024-02-20T02:09:44.206767+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:09:44.206767+0000","last_undegraded":"2024-02-20T02:09:44.206767+0000","last_fullsized":"2024-02-20T02:09:44.206767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-02-20T02:09:45.172870+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:09:45.172870+0000","last_peered":"2024-02-20T02:09:45.172870+0000","last_clean":"2024-02-20T02:09:45.172870+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:09:45.172870+0000","last_undegraded":"2024-02-20T02:09:45.172870+0000","last_fullsized":"2024-02-20T02:09:45.172870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.207654+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:09:44.207654+0000","last_peered":"2024-02-20T02:09:44.207654+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:09:44.207654+0000","last_undegraded":"2024-02-20T02:09:44.207654+0000","last_fullsized":"2024-02-20T02:09:44.207654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-02-20T02:09:45.169513+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:09:45.169513+0000","last_peered":"2024-02-20T02:09:45.169513+0000","last_clean":"2024-02-20T02:09:45.169513+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:09:45.169513+0000","last_undegraded":"2024-02-20T02:09:45.169513+0000","last_fullsized":"2024-02-20T02:09:45.169513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.200438+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:09:44.200438+0000","last_peered":"2024-02-20T02:09:44.200438+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:09:44.200438+0000","last_undegraded":"2024-02-20T02:09:44.200438+0000","last_fullsized":"2024-02-20T02:09:44.200438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.195030+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:09:44.195030+0000","last_peered":"2024-02-20T02:09:44.195030+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:09:44.195030+0000","last_undegraded":"2024-02-20T02:09:44.195030+0000","last_fullsized":"2024-02-20T02:09:44.195030+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.203689+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:09:44.203689+0000","last_peered":"2024-02-20T02:09:44.203689+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:09:44.203689+0000","last_undegraded":"2024-02-20T02:09:44.203689+0000","last_fullsized":"2024-02-20T02:09:44.203689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.214087+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:09:44.214087+0000","last_peered":"2024-02-20T02:09:44.214087+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:09:44.214087+0000","last_undegraded":"2024-02-20T02:09:44.214087+0000","last_fullsized":"2024-02-20T02:09:44.214087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.166285+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:09:44.166285+0000","last_peered":"2024-02-20T02:09:44.166285+0000","last_clean":"2024-02-20T02:09:44.166285+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:09:44.166285+0000","last_undegraded":"2024-02-20T02:09:44.166285+0000","last_fullsized":"2024-02-20T02:09:44.166285+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":712704,"data_stored":620216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:09:46.071 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:09:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.180+0000 7f821be6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.183+0000 7f821be6d640 1 Processor -- start 2024-02-20T02:09:46.182 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.184+0000 7f821be6d640 1 -- start start 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.184+0000 7f821be6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f821409bea0 0x7f8214099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.184+0000 7f821be6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821409b710 con 0x7f821412cd90 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.184+0000 7f821be6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821406eb50 con 0x7f821409bea0 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.184+0000 7f8219c09640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f821409bea0 0x7f8214099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8219408640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f821412cd90 0x7f821412d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36344/0 (socket says 172.21.15.125:36344) 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8219408640 1 -- 172.21.15.125:0/3117131205 learned_addr learned my addr 172.21.15.125:0/3117131205 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8219c09640 1 -- 172.21.15.125:0/3117131205 >> v1:172.21.15.125:6789/0 conn(0x7f821412cd90 legacy=0x7f821412d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8219c09640 1 -- 172.21.15.125:0/3117131205 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821406e070 con 0x7f821409bea0 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8218c07640 1 -- 172.21.15.125:0/3117131205 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124842662 0 0) 0x7f821409b710 con 0x7f821412cd90 2024-02-20T02:09:46.184 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8219c09640 1 --2- 172.21.15.125:0/3117131205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f8214099a70 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaf7a04ca2b8e3a server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8218c07640 1 -- 172.21.15.125:0/3117131205 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8210009ce0 con 0x7f821409bea0 2024-02-20T02:09:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8218c07640 1 -- 172.21.15.125:0/3117131205 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8210009e40 con 0x7f821409bea0 2024-02-20T02:09:46.185 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.185+0000 7f8218c07640 1 -- 172.21.15.125:0/3117131205 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f821000e360 con 0x7f821409bea0 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 -- 172.21.15.125:0/3117131205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 msgr2=0x7f8214099a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 --2- 172.21.15.125:0/3117131205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f8214099a70 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 -- 172.21.15.125:0/3117131205 shutdown_connections 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 --2- 172.21.15.125:0/3117131205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f8214099a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 -- 172.21.15.125:0/3117131205 >> 172.21.15.125:0/3117131205 conn(0x7f821409fae0 msgr2=0x7f821409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 -- 172.21.15.125:0/3117131205 shutdown_connections 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 1 -- 172.21.15.125:0/3117131205 wait complete. 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.186+0000 7f821be6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.187+0000 7f821be6d640 1 Processor -- start 2024-02-20T02:09:46.186 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.187+0000 7f821be6d640 1 -- start start 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f821be6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f821be6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214118cb0 con 0x7f821409bea0 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f8219c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f8219c09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53664/0 (socket says 172.21.15.125:53664) 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f8219c09640 1 -- 172.21.15.125:0/357681927 learned_addr learned my addr 172.21.15.125:0/357681927 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f8219c09640 1 -- 172.21.15.125:0/357681927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821411a3a0 con 0x7f821409bea0 2024-02-20T02:09:46.188 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f8219c09640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8210014410 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8210014570 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821406bf50 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8210014810 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.188+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141189a0 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.189+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f82100149c0 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.189+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214062a30 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.189+0000 7f820a7fc640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 0x7f81f803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.189+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8210019040 con 0x7f821409bea0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.190+0000 7f8219408640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 0x7f81f803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.189 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.190+0000 7f8219408640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 0x7f81f803a3f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.192 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.194+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f821000f990 con 0x7f821409bea0 2024-02-20T02:09:46.363 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.364+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f821412d8b0 con 0x7f81f8037ee0 2024-02-20T02:09:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.365+0000 7f820a7fc640 1 -- 172.21.15.125:0/357681927 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29957 (crc 0 0 0) 0x7f8204001250 con 0x7f81f8037ee0 2024-02-20T02:09:46.364 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:46.364 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 msgr2=0x7f81f803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 0x7f81f803a3f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 msgr2=0x7f82141c4490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 shutdown_connections 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f81f8037ee0 0x7f81f803a3f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 --2- 172.21.15.125:0/357681927 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f821409bea0 0x7f82141c4490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 >> 172.21.15.125:0/357681927 conn(0x7f821409fae0 msgr2=0x7f821412fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:46.367 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 shutdown_connections 2024-02-20T02:09:46.368 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.368+0000 7f821be6d640 1 -- 172.21.15.125:0/357681927 wait complete. 2024-02-20T02:09:46.378 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-02-20T02:09:45.290927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109748,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320652,"statfs":{"total":429496729600,"available":429384347648,"internally_reserved":0,"allocated":1662976,"data_stored":1300128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.206767+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:09:44.206767+0000","last_peered":"2024-02-20T02:09:44.206767+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:09:44.206767+0000","last_undegraded":"2024-02-20T02:09:44.206767+0000","last_fullsized":"2024-02-20T02:09:44.206767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-02-20T02:09:45.172870+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:09:45.172870+0000","last_peered":"2024-02-20T02:09:45.172870+0000","last_clean":"2024-02-20T02:09:45.172870+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:09:45.172870+0000","last_undegraded":"2024-02-20T02:09:45.172870+0000","last_fullsized":"2024-02-20T02:09:45.172870+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.207654+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:09:44.207654+0000","last_peered":"2024-02-20T02:09:44.207654+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:09:44.207654+0000","last_undegraded":"2024-02-20T02:09:44.207654+0000","last_fullsized":"2024-02-20T02:09:44.207654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-02-20T02:09:45.169513+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:09:45.169513+0000","last_peered":"2024-02-20T02:09:45.169513+0000","last_clean":"2024-02-20T02:09:45.169513+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:09:45.169513+0000","last_undegraded":"2024-02-20T02:09:45.169513+0000","last_fullsized":"2024-02-20T02:09:45.169513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.200438+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:09:44.200438+0000","last_peered":"2024-02-20T02:09:44.200438+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:09:44.200438+0000","last_undegraded":"2024-02-20T02:09:44.200438+0000","last_fullsized":"2024-02-20T02:09:44.200438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.195030+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:09:44.195030+0000","last_peered":"2024-02-20T02:09:44.195030+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:09:44.195030+0000","last_undegraded":"2024-02-20T02:09:44.195030+0000","last_fullsized":"2024-02-20T02:09:44.195030+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.203689+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:09:44.203689+0000","last_peered":"2024-02-20T02:09:44.203689+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:09:44.203689+0000","last_undegraded":"2024-02-20T02:09:44.203689+0000","last_fullsized":"2024-02-20T02:09:44.203689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.214087+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:09:44.214087+0000","last_peered":"2024-02-20T02:09:44.214087+0000","last_clean":"2024-02-20T02:09:43.147664+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:09:44.214087+0000","last_undegraded":"2024-02-20T02:09:44.214087+0000","last_fullsized":"2024-02-20T02:09:44.214087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-02-20T02:09:44.166285+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:09:44.166285+0000","last_peered":"2024-02-20T02:09:44.166285+0000","last_clean":"2024-02-20T02:09:44.166285+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:09:44.166285+0000","last_undegraded":"2024-02-20T02:09:44.166285+0000","last_fullsized":"2024-02-20T02:09:44.166285+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27836,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829764,"statfs":{"total":107374182400,"available":107345678336,"internally_reserved":0,"allocated":712704,"data_stored":620216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27068,"kb_used_data":116,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830532,"statfs":{"total":107374182400,"available":107346464768,"internally_reserved":0,"allocated":118784,"data_stored":29848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27776,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829824,"statfs":{"total":107374182400,"available":107345739776,"internally_reserved":0,"allocated":712704,"data_stored":620216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:09:46.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-20T02:09:46.378 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-20T02:09:46.378 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-02-20T02:09:46.379 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-02-20T02:09:46.484 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.483+0000 7f127966d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:46.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.486+0000 7f127966d640 1 Processor -- start 2024-02-20T02:09:46.485 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.486+0000 7f127966d640 1 -- start start 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f127966d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1274134b90 0x7f1274136fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f127966d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1274131820 con 0x7f1274131d00 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f127966d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127409dfd0 con 0x7f1274134b90 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f12727fc640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f1274134b90 0x7f1274136fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f1272ffd640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f1274131d00 0x7f1274132110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:36352/0 (socket says 172.21.15.125:36352) 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f1272ffd640 1 -- 172.21.15.125:0/3368447269 learned_addr learned my addr 172.21.15.125:0/3368447269 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:09:46.486 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f12727fc640 1 -- 172.21.15.125:0/3368447269 >> v1:172.21.15.125:6789/0 conn(0x7f1274131d00 legacy=0x7f1274132110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f1271ffb640 1 -- 172.21.15.125:0/3368447269 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895125754 0 0) 0x7f1274131820 con 0x7f1274131d00 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f12727fc640 1 -- 172.21.15.125:0/3368447269 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274071920 con 0x7f1274134b90 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f12727fc640 1 --2- 172.21.15.125:0/3368447269 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274134b90 0x7f1274136fb0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf782c09d748aa7b server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.487+0000 7f1271ffb640 1 -- 172.21.15.125:0/3368447269 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1268009ce0 con 0x7f1274134b90 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f1271ffb640 1 -- 172.21.15.125:0/3368447269 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1268009e40 con 0x7f1274134b90 2024-02-20T02:09:46.487 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f1271ffb640 1 -- 172.21.15.125:0/3368447269 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f126800e360 con 0x7f1274134b90 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 -- 172.21.15.125:0/3368447269 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274134b90 msgr2=0x7f1274136fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 --2- 172.21.15.125:0/3368447269 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274134b90 0x7f1274136fb0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 -- 172.21.15.125:0/3368447269 shutdown_connections 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 --2- 172.21.15.125:0/3368447269 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274134b90 0x7f1274136fb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 -- 172.21.15.125:0/3368447269 >> 172.21.15.125:0/3368447269 conn(0x7f12740a0c70 msgr2=0x7f12740a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:46.488 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 -- 172.21.15.125:0/3368447269 shutdown_connections 2024-02-20T02:09:46.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 1 -- 172.21.15.125:0/3368447269 wait complete. 2024-02-20T02:09:46.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.488+0000 7f127966d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:09:46.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.489+0000 7f127966d640 1 Processor -- start 2024-02-20T02:09:46.489 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f127966d640 1 -- start start 2024-02-20T02:09:46.490 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f127966d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f127966d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127411fdd0 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:53674/0 (socket says 172.21.15.125:53674) 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f1272ffd640 1 -- 172.21.15.125:0/1456136115 learned_addr learned my addr 172.21.15.125:0/1456136115 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.490+0000 7f1272ffd640 1 -- 172.21.15.125:0/1456136115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127411d140 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f1272ffd640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1264004af0 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1264004c50 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12741c67c0 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1264007120 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.491+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12741c7240 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.492+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f12640072d0 con 0x7f1274131d00 2024-02-20T02:09:46.491 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.492+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127411e920 con 0x7f1274131d00 2024-02-20T02:09:46.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.492+0000 7f125b7fe640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 0x7f126003a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:09:46.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.492+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1264047610 con 0x7f1274131d00 2024-02-20T02:09:46.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.492+0000 7f12727fc640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 0x7f126003a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:09:46.492 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.493+0000 7f12727fc640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 0x7f126003a3f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:09:46.495 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.497+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f126400a9e0 con 0x7f1274131d00 2024-02-20T02:09:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.703+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1274062a30 con 0x7f1274131d00 2024-02-20T02:09:46.702 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.703+0000 7f125b7fe640 1 -- 172.21.15.125:0/1456136115 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (crc 0 0 0) 0x7f126400a5d0 con 0x7f1274131d00 2024-02-20T02:09:46.702 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:09:46.702 INFO:teuthology.orchestra.run.smithi125.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-20T02:09:46.704 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 msgr2=0x7f126003a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.704 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 0x7f126003a3f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.704 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 msgr2=0x7f12741c6b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:09:46.704 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.704 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 shutdown_connections 2024-02-20T02:09:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f1260037ee0 0x7f126003a3f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 --2- 172.21.15.125:0/1456136115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1274131d00 0x7f12741c6b60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:09:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 >> 172.21.15.125:0/1456136115 conn(0x7f12740a0c70 msgr2=0x7f12740a2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:09:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 shutdown_connections 2024-02-20T02:09:46.705 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:09:46.705+0000 7f127966d640 1 -- 172.21.15.125:0/1456136115 wait complete. 2024-02-20T02:09:46.715 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-02-20T02:09:46.715 INFO:teuthology.run_tasks:Running task workunit... 2024-02-20T02:09:46.728 INFO:tasks.workunit:Pulling workunits from ref 663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:09:46.728 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-20T02:09:46.728 DEBUG:teuthology.orchestra.run.smithi125:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-20T02:09:46.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:09:46.743 INFO:teuthology.orchestra.run.smithi125.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-20T02:09:46.743 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-20T02:09:46.797 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-20T02:09:46.798 DEBUG:teuthology.orchestra.run.smithi125:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-20T02:09:46.852 INFO:tasks.workunit:timeout=3h 2024-02-20T02:09:46.852 INFO:tasks.workunit:cleanup=True 2024-02-20T02:09:46.852 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 663dfa8925d03992983e2ae809f7a5fc9835cd6c 2024-02-20T02:09:46.906 INFO:tasks.workunit.client.0.smithi125.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-20T02:10:32.022 INFO:tasks.workunit.client.0.smithi125.stderr:Note: switching to '663dfa8925d03992983e2ae809f7a5fc9835cd6c'. 2024-02-20T02:10:32.022 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.022 INFO:tasks.workunit.client.0.smithi125.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-20T02:10:32.022 INFO:tasks.workunit.client.0.smithi125.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:state without impacting any branches by switching back to a branch. 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: git switch -c 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:Or undo this operation with: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: git switch - 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr: 2024-02-20T02:10:32.023 INFO:tasks.workunit.client.0.smithi125.stderr:HEAD is now at 663dfa8925d Merge branches prs/52267 2024-02-20T02:10:32.026 DEBUG:teuthology.orchestra.run.smithi125:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-20T02:10:32.094 INFO:tasks.workunit.client.0.smithi125.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-20T02:10:32.096 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-20T02:10:32.096 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-20T02:10:32.944 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-20T02:10:33.009 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-20T02:10:33.048 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-20T02:10:33.050 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-20T02:10:33.050 INFO:tasks.workunit.client.0.smithi125.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-20T02:10:33.088 INFO:tasks.workunit.client.0.smithi125.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-20T02:10:33.092 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-20T02:10:33.092 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-20T02:10:33.148 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-20T02:10:33.149 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-20T02:10:33.150 DEBUG:teuthology.orchestra.run.smithi125:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663dfa8925d03992983e2ae809f7a5fc9835cd6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ parallel=1 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' --crimson = --serial ']' 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ crimson=0 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' --crimson = --crimson ']' 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ parallel=0 2024-02-20T02:10:33.208 INFO:tasks.workunit.client.0.smithi125.stderr:+ crimson=1 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ color= 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' -t 1 ']' 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ declare -A pids 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:10:33.209 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_aio 2024-02-20T02:10:33.224 INFO:tasks.workunit.client.0.smithi125.stdout:Running main() from gmock_main.cc 2024-02-20T02:10:33.224 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 42 tests from 2 test suites. 2024-02-20T02:10:33.224 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:10:33.224 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 26 tests from LibRadosAio 2024-02-20T02:10:33.224 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.TooBig 2024-02-20T02:10:35.987 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.TooBig (2763 ms) 2024-02-20T02:10:35.987 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-02-20T02:10:38.999 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleWrite (3012 ms) 2024-02-20T02:10:39.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-02-20T02:10:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.WaitForSafe (3010 ms) 2024-02-20T02:10:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-02-20T02:10:45.026 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTrip (3016 ms) 2024-02-20T02:10:45.026 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-02-20T02:10:48.033 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTrip2 (3007 ms) 2024-02-20T02:10:48.033 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-02-20T02:10:51.059 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTrip3 (3026 ms) 2024-02-20T02:10:51.059 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-02-20T02:10:54.079 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripAppend (3020 ms) 2024-02-20T02:10:54.079 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-02-20T02:10:57.090 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RemoveTest (3012 ms) 2024-02-20T02:10:57.090 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-20T02:11:00.100 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3010 ms) 2024-02-20T02:11:00.100 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RmXattr 2024-02-20T02:11:03.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RmXattr (3010 ms) 2024-02-20T02:11:03.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.XattrIter 2024-02-20T02:11:06.131 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.XattrIter (3021 ms) 2024-02-20T02:11:06.131 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.IsComplete 2024-02-20T02:11:09.145 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.IsComplete (3014 ms) 2024-02-20T02:11:09.146 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.IsSafe 2024-02-20T02:11:12.161 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.IsSafe (3016 ms) 2024-02-20T02:11:12.161 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-02-20T02:11:15.177 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.ReturnValue (3015 ms) 2024-02-20T02:11:15.177 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.Flush 2024-02-20T02:11:18.189 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.Flush (3012 ms) 2024-02-20T02:11:18.189 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-02-20T02:11:21.205 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.FlushAsync (3016 ms) 2024-02-20T02:11:21.205 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-20T02:11:24.214 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3009 ms) 2024-02-20T02:11:24.215 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-20T02:11:26.978 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (2763 ms) 2024-02-20T02:11:26.978 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-02-20T02:11:29.983 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleStat (3006 ms) 2024-02-20T02:11:29.984 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-02-20T02:11:32.990 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.OperateMtime (3007 ms) 2024-02-20T02:11:32.990 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-02-20T02:11:36.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.Operate2Mtime (3009 ms) 2024-02-20T02:11:36.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-02-20T02:11:39.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleStatNS (3012 ms) 2024-02-20T02:11:39.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.StatRemove 2024-02-20T02:11:41.979 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.StatRemove (2967 ms) 2024-02-20T02:11:41.980 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-02-20T02:11:44.992 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.ExecuteClass (3013 ms) 2024-02-20T02:11:44.993 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-02-20T02:11:48.007 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.MultiWrite (3015 ms) 2024-02-20T02:11:48.008 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-02-20T02:11:51.026 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.AioUnlock (3017 ms) 2024-02-20T02:11:51.026 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 26 tests from LibRadosAio (77800 ms total) 2024-02-20T02:11:51.026 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:11:51.026 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 16 tests from LibRadosAioEC 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1100: Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1132: Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1149: Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-02-20T02:11:51.027 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1182: Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1215: Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1286: Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1326: Skipped 2024-02-20T02:11:51.028 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1366: Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.Flush 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1385: Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1415: Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-20T02:11:51.029 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1454: Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1498: Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1532: Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1593: Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-02-20T02:11:51.030 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1649: Skipped 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio.cc:1681: Skipped 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 42 tests from 2 test suites ran. (77800 ms total) 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 26 tests. 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] 16 tests, listed below: 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-02-20T02:11:51.031 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-02-20T02:11:51.032 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-02-20T02:11:51.033 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:11:51.033 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:11:51.033 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_aio_pp 2024-02-20T02:11:51.051 INFO:tasks.workunit.client.0.smithi125.stdout:Running main() from gmock_main.cc 2024-02-20T02:11:51.051 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 57 tests from 4 test suites. 2024-02-20T02:11:51.051 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:11:51.051 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 32 tests from LibRadosAio 2024-02-20T02:11:51.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-02-20T02:11:53.983 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.TooBigPP (2932 ms) 2024-02-20T02:11:53.983 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-02-20T02:12:15.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.PoolQuotaPP (21026 ms) 2024-02-20T02:12:15.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-02-20T02:12:21.046 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleWritePP (6036 ms) 2024-02-20T02:12:21.046 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-02-20T02:12:24.054 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.WaitForSafePP (3009 ms) 2024-02-20T02:12:24.055 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-02-20T02:12:26.988 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripPP (2933 ms) 2024-02-20T02:12:26.988 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-02-20T02:12:29.996 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3009 ms) 2024-02-20T02:12:29.996 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-02-20T02:12:33.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3022 ms) 2024-02-20T02:12:33.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-20T02:12:36.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3011 ms) 2024-02-20T02:12:36.029 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-02-20T02:12:39.040 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.IsCompletePP (3011 ms) 2024-02-20T02:12:39.040 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-02-20T02:12:42.055 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.IsSafePP (3014 ms) 2024-02-20T02:12:42.055 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-02-20T02:12:45.068 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.ReturnValuePP (3014 ms) 2024-02-20T02:12:45.069 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.FlushPP 2024-02-20T02:12:48.077 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.FlushPP (3008 ms) 2024-02-20T02:12:48.077 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-02-20T02:12:51.099 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3022 ms) 2024-02-20T02:12:51.099 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-20T02:12:54.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3014 ms) 2024-02-20T02:12:54.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-20T02:12:56.993 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2881 ms) 2024-02-20T02:12:56.993 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-20T02:12:59.999 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3006 ms) 2024-02-20T02:13:00.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-20T02:13:03.017 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3018 ms) 2024-02-20T02:13:03.017 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-20T02:13:06.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3013 ms) 2024-02-20T02:13:06.030 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-02-20T02:13:09.051 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.SimpleStatPP (3021 ms) 2024-02-20T02:13:09.051 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-02-20T02:13:12.065 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.OperateMtime (3013 ms) 2024-02-20T02:13:12.065 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-02-20T02:13:15.091 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.OperateMtime2 (3027 ms) 2024-02-20T02:13:15.091 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-02-20T02:13:18.101 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.StatRemovePP (3009 ms) 2024-02-20T02:13:18.101 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-02-20T02:13:21.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3013 ms) 2024-02-20T02:13:21.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.OmapPP 2024-02-20T02:13:24.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.OmapPP (3009 ms) 2024-02-20T02:13:24.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-02-20T02:13:27.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.MultiWritePP (2878 ms) 2024-02-20T02:13:27.000 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-02-20T02:13:30.008 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.AioUnlockPP (3008 ms) 2024-02-20T02:13:30.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-20T02:13:33.026 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3018 ms) 2024-02-20T02:13:33.027 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-02-20T02:13:36.042 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RacingRemovePP (3016 ms) 2024-02-20T02:13:36.043 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-20T02:13:39.053 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3011 ms) 2024-02-20T02:13:39.053 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-20T02:13:42.069 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3016 ms) 2024-02-20T02:13:42.069 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-02-20T02:13:44.572 INFO:tasks.workunit.client.0.smithi125.stdout:setting pool EIO 2024-02-20T02:13:54.228 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:13:55.228 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:13:56.228 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:13:57.232 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:13:58.229 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:13:59.229 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:14:00.229 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:14:01.229 INFO:tasks.workunit.client.0.smithi125.stdout:waiting for 499 2024-02-20T02:14:02.229 INFO:tasks.workunit.client.0.smithi125.stdout:max_success 501, min_failed 502 2024-02-20T02:14:03.008 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAio.PoolEIOFlag (20939 ms) 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAio.MultiReads 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:2427: Skipped 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 32 tests from LibRadosAio (131957 ms total) 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 4 tests from LibRadosAioPP 2024-02-20T02:14:03.009 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-20T02:14:06.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3011 ms) 2024-02-20T02:14:06.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-20T02:14:15.020 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (9000 ms) 2024-02-20T02:14:15.020 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-02-20T02:14:30.073 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAioPP.RmXattrPP (15054 ms) 2024-02-20T02:14:30.073 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-20T02:14:33.092 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3019 ms) 2024-02-20T02:14:33.092 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 4 tests from LibRadosAioPP (30084 ms total) 2024-02-20T02:14:33.092 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:33.092 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 1 test from LibRadosIoPP 2024-02-20T02:14:33.092 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.XattrListPP (3014 ms) 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 1 test from LibRadosIoPP (3014 ms total) 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 20 tests from LibRadosAioEC 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1384: Skipped 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-02-20T02:14:36.108 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1419: Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1436: Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1467: Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1500: Skipped 2024-02-20T02:14:36.109 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1605: Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1637: Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1705: Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-02-20T02:14:36.110 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1742: Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1780: Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1796: Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1824: Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.111 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1859: Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1904: Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1946: Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:1977: Skipped 2024-02-20T02:14:36.112 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:2008: Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:2059: Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:2089: Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/aio_cxx.cc:2124: Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:36.113 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 57 tests from 4 test suites ran. (165055 ms total) 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 36 tests. 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] 21 tests, listed below: 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-02-20T02:14:36.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-02-20T02:14:36.115 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-02-20T02:14:36.116 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:14:36.116 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:14:36.116 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_io 2024-02-20T02:14:36.125 INFO:tasks.workunit.client.0.smithi125.stdout:Running main() from gmock_main.cc 2024-02-20T02:14:36.125 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 24 tests from 2 test suites. 2024-02-20T02:14:36.125 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:14:36.125 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 14 tests from LibRadosIo 2024-02-20T02:14:38.114 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-02-20T02:14:38.379 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.SimpleWrite (266 ms) 2024-02-20T02:14:38.379 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.TooBig 2024-02-20T02:14:38.380 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-02-20T02:14:38.380 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-02-20T02:14:38.422 INFO:tasks.workunit.client.0.smithi125.stdout:no timeout :/ 2024-02-20T02:14:38.422 INFO:tasks.workunit.client.0.smithi125.stdout:no timeout :/ 2024-02-20T02:14:38.422 INFO:tasks.workunit.client.0.smithi125.stdout:no timeout :/ 2024-02-20T02:14:38.423 INFO:tasks.workunit.client.0.smithi125.stdout:no timeout :/ 2024-02-20T02:14:38.423 INFO:tasks.workunit.client.0.smithi125.stdout:no timeout :/ 2024-02-20T02:14:38.423 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.ReadTimeout (44 ms) 2024-02-20T02:14:38.423 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-02-20T02:14:38.445 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.RoundTrip (22 ms) 2024-02-20T02:14:38.445 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.Checksum 2024-02-20T02:14:38.456 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.Checksum (11 ms) 2024-02-20T02:14:38.456 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-20T02:14:38.470 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (14 ms) 2024-02-20T02:14:38.470 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-20T02:14:38.484 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (14 ms) 2024-02-20T02:14:38.484 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-02-20T02:14:38.500 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.AppendRoundTrip (17 ms) 2024-02-20T02:14:38.500 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-02-20T02:14:38.503 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-02-20T02:14:38.503 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.TruncTest 2024-02-20T02:14:38.516 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.TruncTest (14 ms) 2024-02-20T02:14:38.517 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-02-20T02:14:38.529 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.RemoveTest (12 ms) 2024-02-20T02:14:38.529 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-20T02:14:38.544 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (15 ms) 2024-02-20T02:14:38.544 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.RmXattr 2024-02-20T02:14:38.584 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.RmXattr (41 ms) 2024-02-20T02:14:38.585 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIo.XattrIter 2024-02-20T02:14:38.603 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIo.XattrIter (18 ms) 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 14 tests from LibRadosIo (490 ms total) 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 10 tests from LibRadosIoEC 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:168: Skipped 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.119 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.120 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.121 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-02-20T02:14:39.122 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:182: Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:196: Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:176: Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-02-20T02:14:39.123 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 24 tests from 2 test suites ran. (2992 ms total) 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 14 tests. 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] 10 tests, listed below: 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-02-20T02:14:39.124 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-02-20T02:14:39.125 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:14:39.125 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:14:39.125 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_io_pp 2024-02-20T02:14:39.135 INFO:tasks.workunit.client.0.smithi125.stdout:Running main() from gmock_main.cc 2024-02-20T02:14:39.135 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 37 tests from 2 test suites. 2024-02-20T02:14:39.135 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:14:39.135 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 20 tests from LibRadosIoPP 2024-02-20T02:14:39.135 INFO:tasks.workunit.client.0.smithi125.stdout:seed 33892 2024-02-20T02:14:41.125 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-02-20T02:14:41.125 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-20T02:14:41.125 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-20T02:14:41.404 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (279 ms) 2024-02-20T02:14:41.404 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-02-20T02:14:41.436 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.ReadOpPP (32 ms) 2024-02-20T02:14:41.436 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-20T02:14:41.447 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (11 ms) 2024-02-20T02:14:41.447 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-02-20T02:14:41.459 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.RoundTripPP (12 ms) 2024-02-20T02:14:41.459 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-20T02:14:41.470 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (11 ms) 2024-02-20T02:14:41.470 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-02-20T02:14:41.478 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.Checksum (9 ms) 2024-02-20T02:14:41.479 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-20T02:14:41.487 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (8 ms) 2024-02-20T02:14:41.487 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-20T02:14:41.503 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (16 ms) 2024-02-20T02:14:41.503 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-20T02:14:41.516 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (13 ms) 2024-02-20T02:14:41.516 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-20T02:14:41.525 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (9 ms) 2024-02-20T02:14:41.525 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-20T02:14:41.538 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (13 ms) 2024-02-20T02:14:41.538 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-02-20T02:14:41.552 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.TruncTestPP (14 ms) 2024-02-20T02:14:41.552 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-20T02:14:41.568 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (16 ms) 2024-02-20T02:14:41.568 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-20T02:14:41.582 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (15 ms) 2024-02-20T02:14:41.583 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-02-20T02:14:41.619 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.RmXattrPP (36 ms) 2024-02-20T02:14:41.619 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-02-20T02:14:41.637 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.XattrListPP (18 ms) 2024-02-20T02:14:41.637 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-02-20T02:14:41.650 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.CmpExtPP (13 ms) 2024-02-20T02:14:41.650 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-20T02:14:41.658 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (8 ms) 2024-02-20T02:14:41.658 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-20T02:14:41.670 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (13 ms) 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 20 tests from LibRadosIoPP (546 ms total) 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 17 tests from LibRadosIoECPP 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:373: Skipped 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.010 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-02-20T02:14:42.011 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.012 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-02-20T02:14:42.013 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.014 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-02-20T02:14:42.015 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.016 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:387: Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:400: Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/testcase_cxx.cc:381: Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:14:42.017 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 37 tests from 2 test suites ran. (2874 ms total) 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 20 tests. 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] 17 tests, listed below: 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-02-20T02:14:42.018 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:14:42.019 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:14:42.020 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_asio 2024-02-20T02:14:42.034 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.037+0000 7f4f6d641640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:42.038 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.041+0000 7f4f6d641640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:42.039 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.041+0000 7f4f6d641640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:42.039 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 12 tests from 1 test suite. 2024-02-20T02:14:42.039 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:14:42.039 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 12 tests from AsioRados 2024-02-20T02:14:42.041 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.041+0000 7f4f6d641640 1 Processor -- start 2024-02-20T02:14:42.041 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.041+0000 7f4f6d641640 1 -- start start 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f6d641640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f6d641640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc78b1abf0 con 0x55bc78c139b0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f66e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f66e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33128/0 (socket says 172.21.15.125:33128) 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f66e4b640 1 -- 172.21.15.125:0/3035459332 learned_addr learned my addr 172.21.15.125:0/3035459332 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f66e4b640 1 -- 172.21.15.125:0/3035459332 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc78abb3c0 con 0x55bc78c139b0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f66e4b640 1 --2- 172.21.15.125:0/3035459332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ee19d9dc0e6a72 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.042+0000 7f4f65648640 1 -- 172.21.15.125:0/3035459332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4f5002ca50 con 0x55bc78c139b0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f65648640 1 -- 172.21.15.125:0/3035459332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4f50035360 con 0x55bc78c139b0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f65648640 1 -- 172.21.15.125:0/3035459332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4f50035600 con 0x55bc78c139b0 2024-02-20T02:14:42.042 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 -- 172.21.15.125:0/3035459332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 msgr2=0x55bc78bff860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3035459332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 -- 172.21.15.125:0/3035459332 shutdown_connections 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3035459332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78bff860 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 -- 172.21.15.125:0/3035459332 >> 172.21.15.125:0/3035459332 conn(0x55bc78a8de00 msgr2=0x55bc78b85a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 -- 172.21.15.125:0/3035459332 shutdown_connections 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.043+0000 7f4f6d641640 1 -- 172.21.15.125:0/3035459332 wait complete. 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f6d641640 1 Processor -- start 2024-02-20T02:14:42.043 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f6d641640 1 -- start start 2024-02-20T02:14:42.045 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f6d641640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:42.045 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f6d641640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc78b163d0 con 0x55bc78c139b0 2024-02-20T02:14:42.045 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f66e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:42.045 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f66e4b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:33140/0 (socket says 172.21.15.125:33140) 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.044+0000 7f4f66e4b640 1 -- 172.21.15.125:0/3125790530 learned_addr learned my addr 172.21.15.125:0/3125790530 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f66e4b640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc78aae570 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f66e4b640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4f5003d040 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4f50035c30 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc78aadfd0 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4f50035ed0 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.045+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc78c03ec0 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f4f50034040 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f5ffff640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 0x7f4f4c038d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4f50031040 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55bc78b0edf0 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=236) v2 ==== 24+0+0 (crc 0 0 0) 0x7f4f50032ee0 con 0x55bc78c139b0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f6664a640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 0x7f4f4c038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:42.046 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.046+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x55bc78b20570 con 0x55bc78c139b0 2024-02-20T02:14:42.047 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:42.047+0000 7f4f6664a640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 0x7f4f4c038d10 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:43.011 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.013+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v237) v1 ==== 43+0+0 (crc 0 0 0) 0x7f4f5003e210 con 0x55bc78c139b0 2024-02-20T02:14:43.012 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.013+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=237}) v3 -- 0x7f4f4c03b180 con 0x55bc78c139b0 2024-02-20T02:14:43.012 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.013+0000 7f4f5ffff640 1 -- 172.21.15.125:0/3125790530 <== mon.0 v2:172.21.15.125:3300/0 8 ==== osd_map(237..237 src has 1..237) v4 ==== 651+0+0 (crc 0 0 0) 0x7f4f5002ee30 con 0x55bc78c139b0 2024-02-20T02:14:43.012 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.014+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 0x55bc78caec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:43.012 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.014+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78c05f80 con 0x55bc78c14300 2024-02-20T02:14:43.012 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.015+0000 7f4f6764c640 1 --2- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 0x55bc78caec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:14:43.017 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:43.020+0000 7f4f6764c640 1 --2- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 0x55bc78caec20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:44.223 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.224+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 1 ==== osd_op_reply(2 exist [writefull 0~5] v238'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.224 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-02-20T02:14:44.224 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.224+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78c02bc0 con 0x55bc78c14300 2024-02-20T02:14:44.224 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.224+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78c03440 con 0x55bc78c14300 2024-02-20T02:14:44.233 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.236+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 2 ==== osd_op_reply(3 exist [read 0~5 out=5b] v238'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.235 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.237+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 3 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.235 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadCallback (13 ms) 2024-02-20T02:14:44.235 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-02-20T02:14:44.236 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.237+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78ca9f20 con 0x55bc78c14300 2024-02-20T02:14:44.236 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.237+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78caf160 con 0x55bc78c14300 2024-02-20T02:14:44.241 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.243+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.241 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.243+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v238'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.241 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadFuture (6 ms) 2024-02-20T02:14:44.242 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-02-20T02:14:44.242 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.243+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78c037f0 con 0x55bc78c14300 2024-02-20T02:14:44.242 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.244+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cefff0 con 0x55bc78c14300 2024-02-20T02:14:44.245 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.247+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 6 ==== osd_op_reply(7 exist [read 0~5 out=5b] v238'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.248 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.250+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 7 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.248 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadYield (7 ms) 2024-02-20T02:14:44.248 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-02-20T02:14:44.248 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.250+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cb04d0 con 0x55bc78c14300 2024-02-20T02:14:44.256 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.258+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 8 ==== osd_op_reply(9 exist [write 0~5] v238'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.256 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteCallback (8 ms) 2024-02-20T02:14:44.256 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-02-20T02:14:44.257 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.258+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cb04d0 con 0x55bc78c14300 2024-02-20T02:14:44.261 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.264+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 9 ==== osd_op_reply(10 exist [write 0~5] v238'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.262 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteFuture (6 ms) 2024-02-20T02:14:44.262 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-02-20T02:14:44.262 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.264+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cd2a20 con 0x55bc78c14300 2024-02-20T02:14:44.267 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.269+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 10 ==== osd_op_reply(11 exist [write 0~5] v238'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.267 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteYield (5 ms) 2024-02-20T02:14:44.267 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-20T02:14:44.268 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.269+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cd2a20 con 0x55bc78c14300 2024-02-20T02:14:44.268 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.269+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cafab0 con 0x55bc78c14300 2024-02-20T02:14:44.270 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.272+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 11 ==== osd_op_reply(12 exist [read 0~5 out=5b] v238'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.270 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.273+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 12 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.271 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (4 ms) 2024-02-20T02:14:44.271 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-20T02:14:44.271 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.273+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78caf610 con 0x55bc78c14300 2024-02-20T02:14:44.271 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.273+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cb0030 con 0x55bc78c14300 2024-02-20T02:14:44.273 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.276+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 13 ==== osd_op_reply(14 exist [read 0~5 out=5b] v238'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.274 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.276+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 14 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f5800c430 con 0x55bc78c14300 2024-02-20T02:14:44.274 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (3 ms) 2024-02-20T02:14:44.274 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-02-20T02:14:44.274 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.276+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78caf610 con 0x55bc78c14300 2024-02-20T02:14:44.274 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.276+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cd30d0 con 0x55bc78c14300 2024-02-20T02:14:44.277 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.279+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 15 ==== osd_op_reply(16 exist [read 0~5 out=5b] v238'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f4f58020040 con 0x55bc78c14300 2024-02-20T02:14:44.277 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.279+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 16 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f4f58020040 con 0x55bc78c14300 2024-02-20T02:14:44.277 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncReadOperationYield (3 ms) 2024-02-20T02:14:44.277 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-20T02:14:44.278 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.279+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78c02fa0 con 0x55bc78c14300 2024-02-20T02:14:44.282 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.285+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 17 ==== osd_op_reply(18 exist [writefull 0~5] v238'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f58020040 con 0x55bc78c14300 2024-02-20T02:14:44.283 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (6 ms) 2024-02-20T02:14:44.283 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-20T02:14:44.283 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.285+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cafb90 con 0x55bc78c14300 2024-02-20T02:14:44.288 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.290+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 18 ==== osd_op_reply(19 exist [writefull 0~5] v238'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f58020040 con 0x55bc78c14300 2024-02-20T02:14:44.288 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (5 ms) 2024-02-20T02:14:44.288 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-20T02:14:44.289 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.290+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 --> v2:172.21.15.125:6804/3877308034 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e237) v8 -- 0x55bc78cafb90 con 0x55bc78c14300 2024-02-20T02:14:44.293 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6764c640 1 -- 172.21.15.125:0/3125790530 <== osd.1 v2:172.21.15.125:6804/3877308034 19 ==== osd_op_reply(20 exist [writefull 0~5] v238'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f4f58020040 con 0x55bc78c14300 2024-02-20T02:14:44.293 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (5 ms) 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 msgr2=0x55bc78caec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 0x55bc78caec20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 msgr2=0x7f4f4c038d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 0x7f4f4c038d10 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 msgr2=0x55bc78ca99e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 shutdown_connections 2024-02-20T02:14:44.294 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> v2:172.21.15.125:6804/3877308034 conn(0x55bc78c14300 0x55bc78caec20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f4f4c036800 0x7f4f4c038d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 --2- 172.21.15.125:0/3125790530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bc78c139b0 0x55bc78ca99e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 >> 172.21.15.125:0/3125790530 conn(0x55bc78a8de00 msgr2=0x55bc78bff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 shutdown_connections 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.295+0000 7f4f6d641640 1 -- 172.21.15.125:0/3125790530 wait complete. 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 12 tests from AsioRados (71 ms total) 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 12 tests from 1 test suite ran. (2255 ms total) 2024-02-20T02:14:44.295 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 12 tests. 2024-02-20T02:14:44.296 INFO:tasks.workunit.client.0.smithi125.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-20T02:14:44.296 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' 0 -eq 1 ']' 2024-02-20T02:14:44.296 INFO:tasks.workunit.client.0.smithi125.stderr:+ ceph_test_rados_api_list 2024-02-20T02:14:44.320 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.322+0000 7fb4c816c940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:44.320 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.322+0000 7fb4c816c940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:44.320 INFO:tasks.workunit.client.0.smithi125.stderr:2024-02-20T02:14:44.322+0000 7fb4c816c940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:44.320 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] Running 11 tests from 3 test suites. 2024-02-20T02:14:44.320 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment set-up. 2024-02-20T02:14:44.321 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 7 tests from LibRadosList 2024-02-20T02:14:46.020 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.ListObjects 2024-02-20T02:14:46.379 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:38: Failure 2024-02-20T02:14:46.380 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:46.380 INFO:tasks.workunit.client.0.smithi125.stdout: std::string(entry) 2024-02-20T02:14:46.380 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: "snapmapper" 2024-02-20T02:14:46.380 INFO:tasks.workunit.client.0.smithi125.stdout: "foo" 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjects (408 ms) 2024-02-20T02:14:46.428 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-20T02:14:46.479 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:56: Failure 2024-02-20T02:14:46.479 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:46.479 INFO:tasks.workunit.client.0.smithi125.stdout: std::string(entry, entry_size) 2024-02-20T02:14:46.479 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: "snapmapper" 2024-02-20T02:14:46.480 INFO:tasks.workunit.client.0.smithi125.stdout: "foo\0bar" 2024-02-20T02:14:46.480 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: "foo" 2024-02-20T02:14:46.527 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:46.528 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:46.528 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:46.528 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:46.528 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:46.528 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsZeroInName (99 ms) 2024-02-20T02:14:46.529 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-02-20T02:14:46.573 INFO:tasks.workunit.client.0.smithi125.stdout:myset foo1,foo2,foo3 2024-02-20T02:14:46.619 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 2024-02-20T02:14:46.619 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:81: Failure 2024-02-20T02:14:46.619 INFO:tasks.workunit.client.0.smithi125.stdout:Value of: myset.end() != myset.find(test_name) 2024-02-20T02:14:46.619 INFO:tasks.workunit.client.0.smithi125.stdout: Actual: false 2024-02-20T02:14:46.620 INFO:tasks.workunit.client.0.smithi125.stdout:Expected: true 2024-02-20T02:14:46.620 INFO:tasks.workunit.client.0.smithi125.stdout:myset foo1,foo4,foo5 2024-02-20T02:14:46.664 INFO:tasks.workunit.client.0.smithi125.stdout:foo4 2024-02-20T02:14:46.664 INFO:tasks.workunit.client.0.smithi125.stdout:foo5 2024-02-20T02:14:46.664 INFO:tasks.workunit.client.0.smithi125.stdout:foo1 2024-02-20T02:14:46.664 INFO:tasks.workunit.client.0.smithi125.stdout:myset foo6,foo7 2024-02-20T02:14:46.708 INFO:tasks.workunit.client.0.smithi125.stdout:foo7 2024-02-20T02:14:46.708 INFO:tasks.workunit.client.0.smithi125.stdout:foo6 2024-02-20T02:14:46.708 INFO:tasks.workunit.client.0.smithi125.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-20T02:14:46.752 INFO:tasks.workunit.client.0.smithi125.stdout::snapmapper 2024-02-20T02:14:46.752 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:81: Failure 2024-02-20T02:14:46.752 INFO:tasks.workunit.client.0.smithi125.stdout:Value of: myset.end() != myset.find(test_name) 2024-02-20T02:14:46.752 INFO:tasks.workunit.client.0.smithi125.stdout: Actual: false 2024-02-20T02:14:46.752 INFO:tasks.workunit.client.0.smithi125.stdout:Expected: true 2024-02-20T02:14:46.801 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:46.801 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:46.801 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:46.802 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:46.802 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:46.802 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsNS (274 ms) 2024-02-20T02:14:46.802 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-02-20T02:14:46.942 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:1 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:10 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:13 0 2024-02-20T02:14:46.943 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:7 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:foo 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:14 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:0 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:15 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:11 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.944 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:5 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:8 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:6 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:foo1 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:foo2 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:3 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:foo3 0 2024-02-20T02:14:46.945 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:4 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:12 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:9 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:snapmapper 0 2024-02-20T02:14:46.946 INFO:tasks.workunit.client.0.smithi125.stdout:2 0 2024-02-20T02:14:46.992 INFO:tasks.workunit.client.0.smithi125.stdout:have snapmapper expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9,foo,foo1,foo2,foo3,snapmapper 2024-02-20T02:14:47.086 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:47.086 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:47.086 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:47.087 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:47.087 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:47.087 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsStart (285 ms) 2024-02-20T02:14:47.087 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-02-20T02:14:47.177 INFO:tasks.workunit.client.0.smithi125.stdout:x cursor=MIN 2024-02-20T02:14:47.224 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.224 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-02-20T02:14:47.224 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-02-20T02:14:47.225 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.226 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=foo1 cursor=76:cd2ae97d:::foo1:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=foo2 cursor=76:cd8c8e4c:::foo2:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=foo3 cursor=76:d071b460:::foo3:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-02-20T02:14:47.227 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.228 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.228 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-02-20T02:14:47.228 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.228 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.228 INFO:tasks.workunit.client.0.smithi125.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-02-20T02:14:47.269 INFO:tasks.workunit.client.0.smithi125.stdout:FIRST> seek to MIN oid=snapmapper 2024-02-20T02:14:47.315 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.315 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.348 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.348 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.348 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.376 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.376 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.376 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:62a1935d:::14:head 2024-02-20T02:14:47.405 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:62a1935d:::14:head -> 14 2024-02-20T02:14:47.405 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.405 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.435 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.435 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.435 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.463 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.463 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.463 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:62a1935d:::14:head 2024-02-20T02:14:47.491 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:62a1935d:::14:head -> 14 2024-02-20T02:14:47.491 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.491 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.520 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.521 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.521 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.549 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.550 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.550 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:62a1935d:::14:head 2024-02-20T02:14:47.578 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:62a1935d:::14:head -> 14 2024-02-20T02:14:47.578 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.578 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.608 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.608 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.608 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.637 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.637 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.637 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:62a1935d:::14:head 2024-02-20T02:14:47.666 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:62a1935d:::14:head -> 14 2024-02-20T02:14:47.666 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.666 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.696 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.696 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.696 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.724 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.725 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-02-20T02:14:47.725 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:62a1935d:::14:head 2024-02-20T02:14:47.753 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:62a1935d:::14:head -> 14 2024-02-20T02:14:47.753 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=snapmapper cursor=76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.753 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:5c867a63:::snapmapper:head 2024-02-20T02:14:47.783 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:5c867a63:::snapmapper:head -> snapmapper 2024-02-20T02:14:47.783 INFO:tasks.workunit.client.0.smithi125.stdout:: oid=foo cursor=76:602f83fe:::foo:head 2024-02-20T02:14:47.783 INFO:tasks.workunit.client.0.smithi125.stdout:: seek to 76:602f83fe:::foo:head 2024-02-20T02:14:47.811 INFO:tasks.workunit.client.0.smithi125.stdout:> 76:602f83fe:::foo:head -> foo 2024-02-20T02:14:47.812 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:250: Failure 2024-02-20T02:14:47.812 INFO:tasks.workunit.client.0.smithi125.stdout:Expected: (count) < (max_objs), actual: 16 vs 16 2024-02-20T02:14:47.861 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:47.861 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:47.862 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:47.862 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:47.862 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:47.862 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsCursor (775 ms) 2024-02-20T02:14:47.862 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-02-20T02:14:47.970 INFO:tasks.workunit.client.0.smithi125.stderr:duplicate obj snapmapper 2024-02-20T02:14:47.970 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:462: Failure 2024-02-20T02:14:47.970 INFO:tasks.workunit.client.0.smithi125.stdout:Value of: saw_obj.count(oid) 2024-02-20T02:14:47.970 INFO:tasks.workunit.client.0.smithi125.stdout: Actual: true 2024-02-20T02:14:47.970 INFO:tasks.workunit.client.0.smithi125.stdout:Expected: false 2024-02-20T02:14:48.019 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.EnumerateObjects (159 ms) 2024-02-20T02:14:48.020 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-20T02:14:48.109 INFO:tasks.workunit.client.0.smithi125.stdout:split 0/5 -> MIN 76:33333333::::head 2024-02-20T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stderr:duplicate obj snapmapper 2024-02-20T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/list.cc:536: Failure 2024-02-20T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stdout:Value of: saw_obj.count(oid) 2024-02-20T02:14:48.120 INFO:tasks.workunit.client.0.smithi125.stdout: Actual: true 2024-02-20T02:14:48.121 INFO:tasks.workunit.client.0.smithi125.stdout:Expected: false 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:61: Failure 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout:Expected equality of these values: 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout: 0 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout: rados_remove(ioctx, entry) 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout: Which is: -2 2024-02-20T02:14:48.171 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.EnumerateObjectsSplit (151 ms) 2024-02-20T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 7 tests from LibRadosList (2151 ms total) 2024-02-20T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 3 tests from LibRadosListEC 2024-02-20T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:70: Skipped 2024-02-20T02:14:49.037 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:84: Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:96: Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:84: Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:96: Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:84: Skipped 2024-02-20T02:14:49.038 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:96: Skipped 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1579-g663dfa89/rpm/el9/BUILD/ceph-19.0.0-1579-g663dfa89/src/test/librados/TestCase.cc:78: Skipped 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:Not supported by crimson yet. Skipped 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 1 test from LibRadosListNP 2024-02-20T02:14:49.039 INFO:tasks.workunit.client.0.smithi125.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-02-20T02:14:52.051 INFO:tasks.workunit.client.0.smithi125.stdout:[ OK ] LibRadosListNP.ListObjectsError (3014 ms) 2024-02-20T02:14:52.051 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] 1 test from LibRadosListNP (3014 ms total) 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[----------] Global test environment tear-down 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[==========] 11 tests from 3 test suites ran. (7731 ms total) 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ PASSED ] 1 test. 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] 3 tests, listed below: 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] 7 tests, listed below: 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjects 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsZeroInName 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsNS 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsStart 2024-02-20T02:14:52.052 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.ListObjectsCursor 2024-02-20T02:14:52.053 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.EnumerateObjects 2024-02-20T02:14:52.053 INFO:tasks.workunit.client.0.smithi125.stdout:[ FAILED ] LibRadosList.EnumerateObjectsSplit 2024-02-20T02:14:52.053 INFO:tasks.workunit.client.0.smithi125.stdout: 2024-02-20T02:14:52.053 INFO:tasks.workunit.client.0.smithi125.stdout: 7 FAILED TESTS 2024-02-20T02:14:52.054 INFO:tasks.workunit.client.0.smithi125.stderr:++ cleanup 2024-02-20T02:14:52.054 INFO:tasks.workunit.client.0.smithi125.stderr:++ pkill -P 32615 2024-02-20T02:14:52.114 INFO:tasks.workunit.client.0.smithi125.stderr:++ true 2024-02-20T02:14:52.114 INFO:tasks.workunit.client.0.smithi125.stderr:+ cleanup 2024-02-20T02:14:52.114 INFO:tasks.workunit.client.0.smithi125.stderr:+ pkill -P 32615 2024-02-20T02:14:52.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:14:52.118 INFO:tasks.workunit.client.0.smithi125.stderr:+ true 2024-02-20T02:14:52.119 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-02-20T02:14:52.119 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-20T02:14:52.503 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi125 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663dfa8925d03992983e2ae809f7a5fc9835cd6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-02-20T02:14:52.715 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bd50b43740404b768caa79b94e0d0d81 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_663dfa8925d03992983e2ae809f7a5fc9835cd6c/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rados/test.sh) on smithi125 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663dfa8925d03992983e2ae809f7a5fc9835cd6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson' 2024-02-20T02:14:52.717 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-02-20T02:14:52.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-20T02:14:52.728 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:14:52.848 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.847+0000 7f8ebf660640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:52.849 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.851+0000 7f8ebf660640 1 Processor -- start 2024-02-20T02:14:52.849 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.851+0000 7f8ebf660640 1 -- start start 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.851+0000 7f8ebf660640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f8eb8133ab0 0x7f8eb8135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.851+0000 7f8ebf660640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb8130840 con 0x7f8eb8130c20 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.851+0000 7f8ebf660640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb8131740 con 0x7f8eb8133ab0 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebcbfb640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f8eb8133ab0 0x7f8eb8135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebd3fc640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f8eb8130c20 0x7f8eb8131030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45536/0 (socket says 172.21.15.125:45536) 2024-02-20T02:14:52.850 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebd3fc640 1 -- 172.21.15.125:0/3785266855 learned_addr learned my addr 172.21.15.125:0/3785266855 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebcbfb640 1 -- 172.21.15.125:0/3785266855 >> v1:172.21.15.125:6789/0 conn(0x7f8eb8130c20 legacy=0x7f8eb8131030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebcbfb640 1 -- 172.21.15.125:0/3785266855 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb806f2f0 con 0x7f8eb8133ab0 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8ebcbfb640 1 --2- 172.21.15.125:0/3785266855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8133ab0 0x7f8eb8135ed0 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6c99ace80bcb25 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8eb7fff640 1 -- 172.21.15.125:0/3785266855 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8eac009ce0 con 0x7f8eb8133ab0 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8eb7fff640 1 -- 172.21.15.125:0/3785266855 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8eac009e40 con 0x7f8eb8133ab0 2024-02-20T02:14:52.851 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.852+0000 7f8eb7fff640 1 -- 172.21.15.125:0/3785266855 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8eac00e360 con 0x7f8eb8133ab0 2024-02-20T02:14:52.853 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 -- 172.21.15.125:0/3785266855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8133ab0 msgr2=0x7f8eb8135ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 --2- 172.21.15.125:0/3785266855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8133ab0 0x7f8eb8135ed0 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 -- 172.21.15.125:0/3785266855 shutdown_connections 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 --2- 172.21.15.125:0/3785266855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8133ab0 0x7f8eb8135ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 -- 172.21.15.125:0/3785266855 >> 172.21.15.125:0/3785266855 conn(0x7f8eb809fae0 msgr2=0x7f8eb809fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 -- 172.21.15.125:0/3785266855 shutdown_connections 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 1 -- 172.21.15.125:0/3785266855 wait complete. 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.853+0000 7f8ebf660640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.854+0000 7f8ebf660640 1 Processor -- start 2024-02-20T02:14:52.854 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.854+0000 7f8ebf660640 1 -- start start 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebf660640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebf660640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb811ef40 con 0x7f8eb8130c20 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebd3fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebd3fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39312/0 (socket says 172.21.15.125:39312) 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebd3fc640 1 -- 172.21.15.125:0/1072005069 learned_addr learned my addr 172.21.15.125:0/1072005069 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebd3fc640 1 -- 172.21.15.125:0/1072005069 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb806bfc0 con 0x7f8eb8130c20 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebd3fc640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:52.855 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8eb00059d0 con 0x7f8eb8130c20 2024-02-20T02:14:52.856 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8eb0005b30 con 0x7f8eb8130c20 2024-02-20T02:14:52.856 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb809a5d0 con 0x7f8eb8130c20 2024-02-20T02:14:52.856 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.855+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8eb0005dd0 con 0x7f8eb8130c20 2024-02-20T02:14:52.856 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.856+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb809a300 con 0x7f8eb8130c20 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.856+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f8eb00070b0 con 0x7f8eb8130c20 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.856+0000 7f8eb5ffb640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 0x7f8ea403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.857+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f8eb00474c0 con 0x7f8eb8130c20 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.857+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb806bfc0 con 0x7f8eb8130c20 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.857+0000 7f8ebcbfb640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 0x7f8ea403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:52.857 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.857+0000 7f8ebcbfb640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 0x7f8ea403a3f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:52.859 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:52.862+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f8eb806bfc0 con 0x7f8eb8130c20 2024-02-20T02:14:53.027 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.029+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8eb80a17b0 con 0x7f8ea4037ee0 2024-02-20T02:14:53.030 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.032+0000 7f8eb5ffb640 1 -- 172.21.15.125:0/1072005069 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107066 (crc 0 0 0) 0x7f8eac012490 con 0x7f8ea4037ee0 2024-02-20T02:14:53.031 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:14:53.031 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:14:53.033 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 msgr2=0x7f8ea403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.033 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 0x7f8ea403a3f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.033 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 msgr2=0x7f8eb811bef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.033 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.033 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 shutdown_connections 2024-02-20T02:14:53.034 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f8ea4037ee0 0x7f8ea403a3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.034 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 --2- 172.21.15.125:0/1072005069 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8eb8130c20 0x7f8eb811bef0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.034 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 >> 172.21.15.125:0/1072005069 conn(0x7f8eb809fae0 msgr2=0x7f8eb81b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.034 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 shutdown_connections 2024-02-20T02:14:53.034 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.034+0000 7f8ebf660640 1 -- 172.21.15.125:0/1072005069 wait complete. 2024-02-20T02:14:53.043 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-02-20T02:14:52.496004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":168096,"kb_used_data":10368,"kb_used_omap":0,"kb_used_meta":157696,"kb_avail":419262304,"statfs":{"total":429496729600,"available":429324599296,"internally_reserved":0,"allocated":10616832,"data_stored":4586898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":161480704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1152,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-24,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001140"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.336365+0000","last_change":"2024-02-20T02:14:44.190037+0000","last_active":"2024-02-20T02:14:48.336365+0000","last_peered":"2024-02-20T02:14:48.336365+0000","last_clean":"2024-02-20T02:14:48.336365+0000","last_became_active":"2024-02-20T02:14:44.170164+0000","last_became_peered":"2024-02-20T02:14:44.170164+0000","last_unstale":"2024-02-20T02:14:48.336365+0000","last_undegraded":"2024-02-20T02:14:48.336365+0000","last_fullsized":"2024-02-20T02:14:48.336365+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.300201+0000","last_change":"2024-02-20T02:14:44.201273+0000","last_active":"2024-02-20T02:14:48.300201+0000","last_peered":"2024-02-20T02:14:48.300201+0000","last_clean":"2024-02-20T02:14:48.300201+0000","last_became_active":"2024-02-20T02:14:44.189022+0000","last_became_peered":"2024-02-20T02:14:44.189022+0000","last_unstale":"2024-02-20T02:14:48.300201+0000","last_undegraded":"2024-02-20T02:14:48.300201+0000","last_fullsized":"2024-02-20T02:14:48.300201+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":18,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.079273+0000","last_change":"2024-02-20T02:14:44.181287+0000","last_active":"2024-02-20T02:14:47.079273+0000","last_peered":"2024-02-20T02:14:47.079273+0000","last_clean":"2024-02-20T02:14:47.079273+0000","last_became_active":"2024-02-20T02:14:44.173755+0000","last_became_peered":"2024-02-20T02:14:44.173755+0000","last_unstale":"2024-02-20T02:14:47.079273+0000","last_undegraded":"2024-02-20T02:14:47.079273+0000","last_fullsized":"2024-02-20T02:14:47.079273+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.280039+0000","last_change":"2024-02-20T02:14:44.211892+0000","last_active":"2024-02-20T02:14:48.280039+0000","last_peered":"2024-02-20T02:14:48.280039+0000","last_clean":"2024-02-20T02:14:48.280039+0000","last_became_active":"2024-02-20T02:14:44.203048+0000","last_became_peered":"2024-02-20T02:14:44.203048+0000","last_unstale":"2024-02-20T02:14:48.280039+0000","last_undegraded":"2024-02-20T02:14:48.280039+0000","last_fullsized":"2024-02-20T02:14:48.280039+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.326420+0000","last_change":"2024-02-20T02:14:44.206069+0000","last_active":"2024-02-20T02:14:48.326420+0000","last_peered":"2024-02-20T02:14:48.326420+0000","last_clean":"2024-02-20T02:14:48.326420+0000","last_became_active":"2024-02-20T02:14:44.201917+0000","last_became_peered":"2024-02-20T02:14:44.201917+0000","last_unstale":"2024-02-20T02:14:48.326420+0000","last_undegraded":"2024-02-20T02:14:48.326420+0000","last_fullsized":"2024-02-20T02:14:48.326420+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292267+0000","last_change":"2024-02-20T02:14:44.187033+0000","last_active":"2024-02-20T02:14:48.292267+0000","last_peered":"2024-02-20T02:14:48.292267+0000","last_clean":"2024-02-20T02:14:48.292267+0000","last_became_active":"2024-02-20T02:14:44.176812+0000","last_became_peered":"2024-02-20T02:14:44.176812+0000","last_unstale":"2024-02-20T02:14:48.292267+0000","last_undegraded":"2024-02-20T02:14:48.292267+0000","last_fullsized":"2024-02-20T02:14:48.292267+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292992+0000","last_change":"2024-02-20T02:14:44.180791+0000","last_active":"2024-02-20T02:14:48.292992+0000","last_peered":"2024-02-20T02:14:48.292992+0000","last_clean":"2024-02-20T02:14:48.292992+0000","last_became_active":"2024-02-20T02:14:44.173660+0000","last_became_peered":"2024-02-20T02:14:44.173660+0000","last_unstale":"2024-02-20T02:14:48.292992+0000","last_undegraded":"2024-02-20T02:14:48.292992+0000","last_fullsized":"2024-02-20T02:14:48.292992+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328854+0000","last_change":"2024-02-20T02:14:44.204673+0000","last_active":"2024-02-20T02:14:48.328854+0000","last_peered":"2024-02-20T02:14:48.328854+0000","last_clean":"2024-02-20T02:14:48.328854+0000","last_became_active":"2024-02-20T02:14:44.193652+0000","last_became_peered":"2024-02-20T02:14:44.193652+0000","last_unstale":"2024-02-20T02:14:48.328854+0000","last_undegraded":"2024-02-20T02:14:48.328854+0000","last_fullsized":"2024-02-20T02:14:48.328854+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.284071+0000","last_change":"2024-02-20T02:14:44.208695+0000","last_active":"2024-02-20T02:14:48.284071+0000","last_peered":"2024-02-20T02:14:48.284071+0000","last_clean":"2024-02-20T02:14:48.284071+0000","last_became_active":"2024-02-20T02:14:44.199375+0000","last_became_peered":"2024-02-20T02:14:44.199375+0000","last_unstale":"2024-02-20T02:14:48.284071+0000","last_undegraded":"2024-02-20T02:14:48.284071+0000","last_fullsized":"2024-02-20T02:14:48.284071+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.063127+0000","last_change":"2024-02-20T02:14:44.203898+0000","last_active":"2024-02-20T02:14:47.063127+0000","last_peered":"2024-02-20T02:14:47.063127+0000","last_clean":"2024-02-20T02:14:47.063127+0000","last_became_active":"2024-02-20T02:14:44.200473+0000","last_became_peered":"2024-02-20T02:14:44.200473+0000","last_unstale":"2024-02-20T02:14:47.063127+0000","last_undegraded":"2024-02-20T02:14:47.063127+0000","last_fullsized":"2024-02-20T02:14:47.063127+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.298985+0000","last_change":"2024-02-20T02:14:44.196513+0000","last_active":"2024-02-20T02:14:48.298985+0000","last_peered":"2024-02-20T02:14:48.298985+0000","last_clean":"2024-02-20T02:14:48.298985+0000","last_became_active":"2024-02-20T02:14:44.190055+0000","last_became_peered":"2024-02-20T02:14:44.190055+0000","last_unstale":"2024-02-20T02:14:48.298985+0000","last_undegraded":"2024-02-20T02:14:48.298985+0000","last_fullsized":"2024-02-20T02:14:48.298985+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.306517+0000","last_change":"2024-02-20T02:14:44.195328+0000","last_active":"2024-02-20T02:14:48.306517+0000","last_peered":"2024-02-20T02:14:48.306517+0000","last_clean":"2024-02-20T02:14:48.306517+0000","last_became_active":"2024-02-20T02:14:44.184534+0000","last_became_peered":"2024-02-20T02:14:44.184534+0000","last_unstale":"2024-02-20T02:14:48.306517+0000","last_undegraded":"2024-02-20T02:14:48.306517+0000","last_fullsized":"2024-02-20T02:14:48.306517+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.278172+0000","last_change":"2024-02-20T02:14:44.201331+0000","last_active":"2024-02-20T02:14:48.278172+0000","last_peered":"2024-02-20T02:14:48.278172+0000","last_clean":"2024-02-20T02:14:48.278172+0000","last_became_active":"2024-02-20T02:14:44.192136+0000","last_became_peered":"2024-02-20T02:14:44.192136+0000","last_unstale":"2024-02-20T02:14:48.278172+0000","last_undegraded":"2024-02-20T02:14:48.278172+0000","last_fullsized":"2024-02-20T02:14:48.278172+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.271811+0000","last_change":"2024-02-20T02:14:44.258170+0000","last_active":"2024-02-20T02:14:48.271811+0000","last_peered":"2024-02-20T02:14:48.271811+0000","last_clean":"2024-02-20T02:14:48.271811+0000","last_became_active":"2024-02-20T02:14:44.254178+0000","last_became_peered":"2024-02-20T02:14:44.254178+0000","last_unstale":"2024-02-20T02:14:48.271811+0000","last_undegraded":"2024-02-20T02:14:48.271811+0000","last_fullsized":"2024-02-20T02:14:48.271811+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.319527+0000","last_change":"2024-02-20T02:14:44.253515+0000","last_active":"2024-02-20T02:14:48.319527+0000","last_peered":"2024-02-20T02:14:48.319527+0000","last_clean":"2024-02-20T02:14:48.319527+0000","last_became_active":"2024-02-20T02:14:44.250168+0000","last_became_peered":"2024-02-20T02:14:44.250168+0000","last_unstale":"2024-02-20T02:14:48.319527+0000","last_undegraded":"2024-02-20T02:14:48.319527+0000","last_fullsized":"2024-02-20T02:14:48.319527+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.316346+0000","last_change":"2024-02-20T02:14:44.212643+0000","last_active":"2024-02-20T02:14:48.316346+0000","last_peered":"2024-02-20T02:14:48.316346+0000","last_clean":"2024-02-20T02:14:48.316346+0000","last_became_active":"2024-02-20T02:14:44.208652+0000","last_became_peered":"2024-02-20T02:14:44.208652+0000","last_unstale":"2024-02-20T02:14:48.316346+0000","last_undegraded":"2024-02-20T02:14:48.316346+0000","last_fullsized":"2024-02-20T02:14:48.316346+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.309446+0000","last_change":"2024-02-20T02:14:44.206611+0000","last_active":"2024-02-20T02:14:48.309446+0000","last_peered":"2024-02-20T02:14:48.309446+0000","last_clean":"2024-02-20T02:14:48.309446+0000","last_became_active":"2024-02-20T02:14:44.197136+0000","last_became_peered":"2024-02-20T02:14:44.197136+0000","last_unstale":"2024-02-20T02:14:48.309446+0000","last_undegraded":"2024-02-20T02:14:48.309446+0000","last_fullsized":"2024-02-20T02:14:48.309446+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.303496+0000","last_change":"2024-02-20T02:14:44.224136+0000","last_active":"2024-02-20T02:14:48.303496+0000","last_peered":"2024-02-20T02:14:48.303496+0000","last_clean":"2024-02-20T02:14:48.303496+0000","last_became_active":"2024-02-20T02:14:44.218175+0000","last_became_peered":"2024-02-20T02:14:44.218175+0000","last_unstale":"2024-02-20T02:14:48.303496+0000","last_undegraded":"2024-02-20T02:14:48.303496+0000","last_fullsized":"2024-02-20T02:14:48.303496+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.078198+0000","last_change":"2024-02-20T02:14:44.182566+0000","last_active":"2024-02-20T02:14:47.078198+0000","last_peered":"2024-02-20T02:14:47.078198+0000","last_clean":"2024-02-20T02:14:47.078198+0000","last_became_active":"2024-02-20T02:14:44.175757+0000","last_became_peered":"2024-02-20T02:14:44.175757+0000","last_unstale":"2024-02-20T02:14:47.078198+0000","last_undegraded":"2024-02-20T02:14:47.078198+0000","last_fullsized":"2024-02-20T02:14:47.078198+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.350432+0000","last_change":"2024-02-20T02:14:44.179208+0000","last_active":"2024-02-20T02:14:48.350432+0000","last_peered":"2024-02-20T02:14:48.350432+0000","last_clean":"2024-02-20T02:14:48.350432+0000","last_became_active":"2024-02-20T02:14:44.167479+0000","last_became_peered":"2024-02-20T02:14:44.167479+0000","last_unstale":"2024-02-20T02:14:48.350432+0000","last_undegraded":"2024-02-20T02:14:48.350432+0000","last_fullsized":"2024-02-20T02:14:48.350432+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.338096+0000","last_change":"2024-02-20T02:14:44.192711+0000","last_active":"2024-02-20T02:14:48.338096+0000","last_peered":"2024-02-20T02:14:48.338096+0000","last_clean":"2024-02-20T02:14:48.338096+0000","last_became_active":"2024-02-20T02:14:44.180769+0000","last_became_peered":"2024-02-20T02:14:44.180769+0000","last_unstale":"2024-02-20T02:14:48.338096+0000","last_undegraded":"2024-02-20T02:14:48.338096+0000","last_fullsized":"2024-02-20T02:14:48.338096+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.327481+0000","last_change":"2024-02-20T02:14:44.191580+0000","last_active":"2024-02-20T02:14:48.327481+0000","last_peered":"2024-02-20T02:14:48.327481+0000","last_clean":"2024-02-20T02:14:48.327481+0000","last_became_active":"2024-02-20T02:14:44.182801+0000","last_became_peered":"2024-02-20T02:14:44.182801+0000","last_unstale":"2024-02-20T02:14:48.327481+0000","last_undegraded":"2024-02-20T02:14:48.327481+0000","last_fullsized":"2024-02-20T02:14:48.327481+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.060000+0000","last_change":"2024-02-20T02:14:44.190778+0000","last_active":"2024-02-20T02:14:47.060000+0000","last_peered":"2024-02-20T02:14:47.060000+0000","last_clean":"2024-02-20T02:14:47.060000+0000","last_became_active":"2024-02-20T02:14:44.186481+0000","last_became_peered":"2024-02-20T02:14:44.186481+0000","last_unstale":"2024-02-20T02:14:47.060000+0000","last_undegraded":"2024-02-20T02:14:47.060000+0000","last_fullsized":"2024-02-20T02:14:47.060000+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.259046+0000","last_change":"2024-02-20T02:14:44.246778+0000","last_active":"2024-02-20T02:14:48.259046+0000","last_peered":"2024-02-20T02:14:48.259046+0000","last_clean":"2024-02-20T02:14:48.259046+0000","last_became_active":"2024-02-20T02:14:44.243754+0000","last_became_peered":"2024-02-20T02:14:44.243754+0000","last_unstale":"2024-02-20T02:14:48.259046+0000","last_undegraded":"2024-02-20T02:14:48.259046+0000","last_fullsized":"2024-02-20T02:14:48.259046+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.267425+0000","last_change":"2024-02-20T02:14:44.205105+0000","last_active":"2024-02-20T02:14:48.267425+0000","last_peered":"2024-02-20T02:14:48.267425+0000","last_clean":"2024-02-20T02:14:48.267425+0000","last_became_active":"2024-02-20T02:14:44.194528+0000","last_became_peered":"2024-02-20T02:14:44.194528+0000","last_unstale":"2024-02-20T02:14:48.267425+0000","last_undegraded":"2024-02-20T02:14:48.267425+0000","last_fullsized":"2024-02-20T02:14:48.267425+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.312011+0000","last_change":"2024-02-20T02:14:44.248214+0000","last_active":"2024-02-20T02:14:48.312011+0000","last_peered":"2024-02-20T02:14:48.312011+0000","last_clean":"2024-02-20T02:14:48.312011+0000","last_became_active":"2024-02-20T02:14:44.244704+0000","last_became_peered":"2024-02-20T02:14:44.244704+0000","last_unstale":"2024-02-20T02:14:48.312011+0000","last_undegraded":"2024-02-20T02:14:48.312011+0000","last_fullsized":"2024-02-20T02:14:48.312011+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.295402+0000","last_change":"2024-02-20T02:14:44.238032+0000","last_active":"2024-02-20T02:14:48.295402+0000","last_peered":"2024-02-20T02:14:48.295402+0000","last_clean":"2024-02-20T02:14:48.295402+0000","last_became_active":"2024-02-20T02:14:44.234764+0000","last_became_peered":"2024-02-20T02:14:44.234764+0000","last_unstale":"2024-02-20T02:14:48.295402+0000","last_undegraded":"2024-02-20T02:14:48.295402+0000","last_fullsized":"2024-02-20T02:14:48.295402+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.308248+0000","last_change":"2024-02-20T02:14:44.241835+0000","last_active":"2024-02-20T02:14:48.308248+0000","last_peered":"2024-02-20T02:14:48.308248+0000","last_clean":"2024-02-20T02:14:48.308248+0000","last_became_active":"2024-02-20T02:14:44.238057+0000","last_became_peered":"2024-02-20T02:14:44.238057+0000","last_unstale":"2024-02-20T02:14:48.308248+0000","last_undegraded":"2024-02-20T02:14:48.308248+0000","last_fullsized":"2024-02-20T02:14:48.308248+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.317991+0000","last_change":"2024-02-20T02:14:44.199397+0000","last_active":"2024-02-20T02:14:48.317991+0000","last_peered":"2024-02-20T02:14:48.317991+0000","last_clean":"2024-02-20T02:14:48.317991+0000","last_became_active":"2024-02-20T02:14:44.182848+0000","last_became_peered":"2024-02-20T02:14:44.182848+0000","last_unstale":"2024-02-20T02:14:48.317991+0000","last_undegraded":"2024-02-20T02:14:48.317991+0000","last_fullsized":"2024-02-20T02:14:48.317991+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328213+0000","last_change":"2024-02-20T02:14:44.202362+0000","last_active":"2024-02-20T02:14:48.328213+0000","last_peered":"2024-02-20T02:14:48.328213+0000","last_clean":"2024-02-20T02:14:48.328213+0000","last_became_active":"2024-02-20T02:14:44.191242+0000","last_became_peered":"2024-02-20T02:14:44.191242+0000","last_unstale":"2024-02-20T02:14:48.328213+0000","last_undegraded":"2024-02-20T02:14:48.328213+0000","last_fullsized":"2024-02-20T02:14:48.328213+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"238'7","reported_seq":30,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349794+0000","last_change":"2024-02-20T02:14:44.187038+0000","last_active":"2024-02-20T02:14:48.349794+0000","last_peered":"2024-02-20T02:14:48.349794+0000","last_clean":"2024-02-20T02:14:48.349794+0000","last_became_active":"2024-02-20T02:14:44.172077+0000","last_became_peered":"2024-02-20T02:14:44.172077+0000","last_unstale":"2024-02-20T02:14:48.349794+0000","last_undegraded":"2024-02-20T02:14:48.349794+0000","last_fullsized":"2024-02-20T02:14:48.349794+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.068690+0000","last_change":"2024-02-20T02:14:44.178106+0000","last_active":"2024-02-20T02:14:47.068690+0000","last_peered":"2024-02-20T02:14:47.068690+0000","last_clean":"2024-02-20T02:14:47.068690+0000","last_became_active":"2024-02-20T02:14:44.171906+0000","last_became_peered":"2024-02-20T02:14:44.171906+0000","last_unstale":"2024-02-20T02:14:47.068690+0000","last_undegraded":"2024-02-20T02:14:47.068690+0000","last_fullsized":"2024-02-20T02:14:47.068690+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.351035+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:14:48.351035+0000","last_peered":"2024-02-20T02:14:48.351035+0000","last_clean":"2024-02-20T02:14:48.351035+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:14:48.351035+0000","last_undegraded":"2024-02-20T02:14:48.351035+0000","last_fullsized":"2024-02-20T02:14:48.351035+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.064824+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:14:47.064824+0000","last_peered":"2024-02-20T02:14:47.064824+0000","last_clean":"2024-02-20T02:14:47.064824+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:14:47.064824+0000","last_undegraded":"2024-02-20T02:14:47.064824+0000","last_fullsized":"2024-02-20T02:14:47.064824+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":609,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.299517+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:14:48.299517+0000","last_peered":"2024-02-20T02:14:48.299517+0000","last_clean":"2024-02-20T02:14:48.299517+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:14:48.299517+0000","last_undegraded":"2024-02-20T02:14:48.299517+0000","last_fullsized":"2024-02-20T02:14:48.299517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.071227+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:14:47.071227+0000","last_peered":"2024-02-20T02:14:47.071227+0000","last_clean":"2024-02-20T02:14:47.071227+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:14:47.071227+0000","last_undegraded":"2024-02-20T02:14:47.071227+0000","last_fullsized":"2024-02-20T02:14:47.071227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.289711+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:14:48.289711+0000","last_peered":"2024-02-20T02:14:48.289711+0000","last_clean":"2024-02-20T02:14:48.289711+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:14:48.289711+0000","last_undegraded":"2024-02-20T02:14:48.289711+0000","last_fullsized":"2024-02-20T02:14:48.289711+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349125+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:14:48.349125+0000","last_peered":"2024-02-20T02:14:48.349125+0000","last_clean":"2024-02-20T02:14:48.349125+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:14:48.349125+0000","last_undegraded":"2024-02-20T02:14:48.349125+0000","last_fullsized":"2024-02-20T02:14:48.349125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.325381+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:14:48.325381+0000","last_peered":"2024-02-20T02:14:48.325381+0000","last_clean":"2024-02-20T02:14:48.325381+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:14:48.325381+0000","last_undegraded":"2024-02-20T02:14:48.325381+0000","last_fullsized":"2024-02-20T02:14:48.325381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.367387+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:14:48.367387+0000","last_peered":"2024-02-20T02:14:48.367387+0000","last_clean":"2024-02-20T02:14:48.367387+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:14:48.367387+0000","last_undegraded":"2024-02-20T02:14:48.367387+0000","last_fullsized":"2024-02-20T02:14:48.367387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":745,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.269283+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:14:48.269283+0000","last_peered":"2024-02-20T02:14:48.269283+0000","last_clean":"2024-02-20T02:14:48.269283+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:14:48.269283+0000","last_undegraded":"2024-02-20T02:14:48.269283+0000","last_fullsized":"2024-02-20T02:14:48.269283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":42360,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815240,"statfs":{"total":107374182400,"available":107330805760,"internally_reserved":0,"allocated":2998272,"data_stored":1458322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":37576,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":35264,"kb_avail":104820024,"statfs":{"total":107374182400,"available":107335704576,"internally_reserved":0,"allocated":2359296,"data_stored":846583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36110336},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":41700,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815900,"statfs":{"total":107374182400,"available":107331481600,"internally_reserved":0,"allocated":2322432,"data_stored":845431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":46460,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":43584,"kb_avail":104811140,"statfs":{"total":107374182400,"available":107326607360,"internally_reserved":0,"allocated":2936832,"data_stored":1436562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44630016},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:14:53.045 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:14:53.150 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.149+0000 7f9f3da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.150 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f3da6d640 1 Processor -- start 2024-02-20T02:14:53.151 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f3da6d640 1 -- start start 2024-02-20T02:14:53.153 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f3da6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f9f38134ba0 0x7f9f38136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.153 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f3da6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f38131890 con 0x7f9f38131c50 2024-02-20T02:14:53.153 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f3da6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f38072db0 con 0x7f9f38134ba0 2024-02-20T02:14:53.153 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f36ffd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f9f38134ba0 0x7f9f38136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.153 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.153+0000 7f9f377fe640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f9f38131c50 0x7f9f38132060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45548/0 (socket says 172.21.15.125:45548) 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f377fe640 1 -- 172.21.15.125:0/2477683207 learned_addr learned my addr 172.21.15.125:0/2477683207 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f36ffd640 1 -- 172.21.15.125:0/2477683207 >> v1:172.21.15.125:6789/0 conn(0x7f9f38131c50 legacy=0x7f9f38132060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f367fc640 1 -- 172.21.15.125:0/2477683207 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21943132 0 0) 0x7f9f38131890 con 0x7f9f38131c50 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f36ffd640 1 -- 172.21.15.125:0/2477683207 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3809c700 con 0x7f9f38134ba0 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f36ffd640 1 --2- 172.21.15.125:0/2477683207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38134ba0 0x7f9f38136fc0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6a1a6d0e342680 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f367fc640 1 -- 172.21.15.125:0/2477683207 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9f2c009ce0 con 0x7f9f38134ba0 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f367fc640 1 -- 172.21.15.125:0/2477683207 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9f2c009e40 con 0x7f9f38134ba0 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.154+0000 7f9f367fc640 1 -- 172.21.15.125:0/2477683207 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9f2c00e360 con 0x7f9f38134ba0 2024-02-20T02:14:53.154 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 -- 172.21.15.125:0/2477683207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38134ba0 msgr2=0x7f9f38136fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/2477683207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38134ba0 0x7f9f38136fc0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 -- 172.21.15.125:0/2477683207 shutdown_connections 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/2477683207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38134ba0 0x7f9f38136fc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 -- 172.21.15.125:0/2477683207 >> 172.21.15.125:0/2477683207 conn(0x7f9f3809f890 msgr2=0x7f9f3809fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 -- 172.21.15.125:0/2477683207 shutdown_connections 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 1 -- 172.21.15.125:0/2477683207 wait complete. 2024-02-20T02:14:53.155 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.155+0000 7f9f3da6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.156+0000 7f9f3da6d640 1 Processor -- start 2024-02-20T02:14:53.157 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.156+0000 7f9f3da6d640 1 -- start start 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f3da6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f3da6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f381c6300 con 0x7f9f38131c50 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39338/0 (socket says 172.21.15.125:39338) 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f377fe640 1 -- 172.21.15.125:0/834998073 learned_addr learned my addr 172.21.15.125:0/834998073 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f377fe640 1 -- 172.21.15.125:0/834998073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3811e140 con 0x7f9f38131c50 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f377fe640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9f28004af0 con 0x7f9f38131c50 2024-02-20T02:14:53.158 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9f28004c50 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f381c65a0 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9f28007120 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.157+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3811e8d0 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.158+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f9f280072d0 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.158+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f38062a30 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.158+0000 7f9f1ffff640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 0x7f9f2403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.158+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f9f28047700 con 0x7f9f38131c50 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.159+0000 7f9f36ffd640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 0x7f9f2403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.159 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.159+0000 7f9f36ffd640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 0x7f9f2403a3f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.161 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.163+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f9f2800a9e0 con 0x7f9f38131c50 2024-02-20T02:14:53.332 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.333+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f3809e9b0 con 0x7f9f24037ee0 2024-02-20T02:14:53.335 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.336+0000 7f9f1ffff640 1 -- 172.21.15.125:0/834998073 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107066 (crc 0 0 0) 0x7f9f2c009cb0 con 0x7f9f24037ee0 2024-02-20T02:14:53.336 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:14:53.336 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:14:53.338 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.338+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 msgr2=0x7f9f2403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.338 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.338+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 0x7f9f2403a3f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.338 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 msgr2=0x7f9f381c6940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.338 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.338 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 shutdown_connections 2024-02-20T02:14:53.339 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f9f24037ee0 0x7f9f2403a3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.339 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 --2- 172.21.15.125:0/834998073 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f38131c50 0x7f9f381c6940 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.339 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 >> 172.21.15.125:0/834998073 conn(0x7f9f3809f890 msgr2=0x7f9f380a1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.339 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 shutdown_connections 2024-02-20T02:14:53.339 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.339+0000 7f9f3da6d640 1 -- 172.21.15.125:0/834998073 wait complete. 2024-02-20T02:14:53.348 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-02-20T02:14:52.496004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":168096,"kb_used_data":10368,"kb_used_omap":0,"kb_used_meta":157696,"kb_avail":419262304,"statfs":{"total":429496729600,"available":429324599296,"internally_reserved":0,"allocated":10616832,"data_stored":4586898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":161480704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1152,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-24,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001140"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.336365+0000","last_change":"2024-02-20T02:14:44.190037+0000","last_active":"2024-02-20T02:14:48.336365+0000","last_peered":"2024-02-20T02:14:48.336365+0000","last_clean":"2024-02-20T02:14:48.336365+0000","last_became_active":"2024-02-20T02:14:44.170164+0000","last_became_peered":"2024-02-20T02:14:44.170164+0000","last_unstale":"2024-02-20T02:14:48.336365+0000","last_undegraded":"2024-02-20T02:14:48.336365+0000","last_fullsized":"2024-02-20T02:14:48.336365+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.300201+0000","last_change":"2024-02-20T02:14:44.201273+0000","last_active":"2024-02-20T02:14:48.300201+0000","last_peered":"2024-02-20T02:14:48.300201+0000","last_clean":"2024-02-20T02:14:48.300201+0000","last_became_active":"2024-02-20T02:14:44.189022+0000","last_became_peered":"2024-02-20T02:14:44.189022+0000","last_unstale":"2024-02-20T02:14:48.300201+0000","last_undegraded":"2024-02-20T02:14:48.300201+0000","last_fullsized":"2024-02-20T02:14:48.300201+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":18,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.079273+0000","last_change":"2024-02-20T02:14:44.181287+0000","last_active":"2024-02-20T02:14:47.079273+0000","last_peered":"2024-02-20T02:14:47.079273+0000","last_clean":"2024-02-20T02:14:47.079273+0000","last_became_active":"2024-02-20T02:14:44.173755+0000","last_became_peered":"2024-02-20T02:14:44.173755+0000","last_unstale":"2024-02-20T02:14:47.079273+0000","last_undegraded":"2024-02-20T02:14:47.079273+0000","last_fullsized":"2024-02-20T02:14:47.079273+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.280039+0000","last_change":"2024-02-20T02:14:44.211892+0000","last_active":"2024-02-20T02:14:48.280039+0000","last_peered":"2024-02-20T02:14:48.280039+0000","last_clean":"2024-02-20T02:14:48.280039+0000","last_became_active":"2024-02-20T02:14:44.203048+0000","last_became_peered":"2024-02-20T02:14:44.203048+0000","last_unstale":"2024-02-20T02:14:48.280039+0000","last_undegraded":"2024-02-20T02:14:48.280039+0000","last_fullsized":"2024-02-20T02:14:48.280039+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.326420+0000","last_change":"2024-02-20T02:14:44.206069+0000","last_active":"2024-02-20T02:14:48.326420+0000","last_peered":"2024-02-20T02:14:48.326420+0000","last_clean":"2024-02-20T02:14:48.326420+0000","last_became_active":"2024-02-20T02:14:44.201917+0000","last_became_peered":"2024-02-20T02:14:44.201917+0000","last_unstale":"2024-02-20T02:14:48.326420+0000","last_undegraded":"2024-02-20T02:14:48.326420+0000","last_fullsized":"2024-02-20T02:14:48.326420+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292267+0000","last_change":"2024-02-20T02:14:44.187033+0000","last_active":"2024-02-20T02:14:48.292267+0000","last_peered":"2024-02-20T02:14:48.292267+0000","last_clean":"2024-02-20T02:14:48.292267+0000","last_became_active":"2024-02-20T02:14:44.176812+0000","last_became_peered":"2024-02-20T02:14:44.176812+0000","last_unstale":"2024-02-20T02:14:48.292267+0000","last_undegraded":"2024-02-20T02:14:48.292267+0000","last_fullsized":"2024-02-20T02:14:48.292267+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292992+0000","last_change":"2024-02-20T02:14:44.180791+0000","last_active":"2024-02-20T02:14:48.292992+0000","last_peered":"2024-02-20T02:14:48.292992+0000","last_clean":"2024-02-20T02:14:48.292992+0000","last_became_active":"2024-02-20T02:14:44.173660+0000","last_became_peered":"2024-02-20T02:14:44.173660+0000","last_unstale":"2024-02-20T02:14:48.292992+0000","last_undegraded":"2024-02-20T02:14:48.292992+0000","last_fullsized":"2024-02-20T02:14:48.292992+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328854+0000","last_change":"2024-02-20T02:14:44.204673+0000","last_active":"2024-02-20T02:14:48.328854+0000","last_peered":"2024-02-20T02:14:48.328854+0000","last_clean":"2024-02-20T02:14:48.328854+0000","last_became_active":"2024-02-20T02:14:44.193652+0000","last_became_peered":"2024-02-20T02:14:44.193652+0000","last_unstale":"2024-02-20T02:14:48.328854+0000","last_undegraded":"2024-02-20T02:14:48.328854+0000","last_fullsized":"2024-02-20T02:14:48.328854+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.284071+0000","last_change":"2024-02-20T02:14:44.208695+0000","last_active":"2024-02-20T02:14:48.284071+0000","last_peered":"2024-02-20T02:14:48.284071+0000","last_clean":"2024-02-20T02:14:48.284071+0000","last_became_active":"2024-02-20T02:14:44.199375+0000","last_became_peered":"2024-02-20T02:14:44.199375+0000","last_unstale":"2024-02-20T02:14:48.284071+0000","last_undegraded":"2024-02-20T02:14:48.284071+0000","last_fullsized":"2024-02-20T02:14:48.284071+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.063127+0000","last_change":"2024-02-20T02:14:44.203898+0000","last_active":"2024-02-20T02:14:47.063127+0000","last_peered":"2024-02-20T02:14:47.063127+0000","last_clean":"2024-02-20T02:14:47.063127+0000","last_became_active":"2024-02-20T02:14:44.200473+0000","last_became_peered":"2024-02-20T02:14:44.200473+0000","last_unstale":"2024-02-20T02:14:47.063127+0000","last_undegraded":"2024-02-20T02:14:47.063127+0000","last_fullsized":"2024-02-20T02:14:47.063127+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.298985+0000","last_change":"2024-02-20T02:14:44.196513+0000","last_active":"2024-02-20T02:14:48.298985+0000","last_peered":"2024-02-20T02:14:48.298985+0000","last_clean":"2024-02-20T02:14:48.298985+0000","last_became_active":"2024-02-20T02:14:44.190055+0000","last_became_peered":"2024-02-20T02:14:44.190055+0000","last_unstale":"2024-02-20T02:14:48.298985+0000","last_undegraded":"2024-02-20T02:14:48.298985+0000","last_fullsized":"2024-02-20T02:14:48.298985+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.306517+0000","last_change":"2024-02-20T02:14:44.195328+0000","last_active":"2024-02-20T02:14:48.306517+0000","last_peered":"2024-02-20T02:14:48.306517+0000","last_clean":"2024-02-20T02:14:48.306517+0000","last_became_active":"2024-02-20T02:14:44.184534+0000","last_became_peered":"2024-02-20T02:14:44.184534+0000","last_unstale":"2024-02-20T02:14:48.306517+0000","last_undegraded":"2024-02-20T02:14:48.306517+0000","last_fullsized":"2024-02-20T02:14:48.306517+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.278172+0000","last_change":"2024-02-20T02:14:44.201331+0000","last_active":"2024-02-20T02:14:48.278172+0000","last_peered":"2024-02-20T02:14:48.278172+0000","last_clean":"2024-02-20T02:14:48.278172+0000","last_became_active":"2024-02-20T02:14:44.192136+0000","last_became_peered":"2024-02-20T02:14:44.192136+0000","last_unstale":"2024-02-20T02:14:48.278172+0000","last_undegraded":"2024-02-20T02:14:48.278172+0000","last_fullsized":"2024-02-20T02:14:48.278172+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.271811+0000","last_change":"2024-02-20T02:14:44.258170+0000","last_active":"2024-02-20T02:14:48.271811+0000","last_peered":"2024-02-20T02:14:48.271811+0000","last_clean":"2024-02-20T02:14:48.271811+0000","last_became_active":"2024-02-20T02:14:44.254178+0000","last_became_peered":"2024-02-20T02:14:44.254178+0000","last_unstale":"2024-02-20T02:14:48.271811+0000","last_undegraded":"2024-02-20T02:14:48.271811+0000","last_fullsized":"2024-02-20T02:14:48.271811+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.319527+0000","last_change":"2024-02-20T02:14:44.253515+0000","last_active":"2024-02-20T02:14:48.319527+0000","last_peered":"2024-02-20T02:14:48.319527+0000","last_clean":"2024-02-20T02:14:48.319527+0000","last_became_active":"2024-02-20T02:14:44.250168+0000","last_became_peered":"2024-02-20T02:14:44.250168+0000","last_unstale":"2024-02-20T02:14:48.319527+0000","last_undegraded":"2024-02-20T02:14:48.319527+0000","last_fullsized":"2024-02-20T02:14:48.319527+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.316346+0000","last_change":"2024-02-20T02:14:44.212643+0000","last_active":"2024-02-20T02:14:48.316346+0000","last_peered":"2024-02-20T02:14:48.316346+0000","last_clean":"2024-02-20T02:14:48.316346+0000","last_became_active":"2024-02-20T02:14:44.208652+0000","last_became_peered":"2024-02-20T02:14:44.208652+0000","last_unstale":"2024-02-20T02:14:48.316346+0000","last_undegraded":"2024-02-20T02:14:48.316346+0000","last_fullsized":"2024-02-20T02:14:48.316346+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.309446+0000","last_change":"2024-02-20T02:14:44.206611+0000","last_active":"2024-02-20T02:14:48.309446+0000","last_peered":"2024-02-20T02:14:48.309446+0000","last_clean":"2024-02-20T02:14:48.309446+0000","last_became_active":"2024-02-20T02:14:44.197136+0000","last_became_peered":"2024-02-20T02:14:44.197136+0000","last_unstale":"2024-02-20T02:14:48.309446+0000","last_undegraded":"2024-02-20T02:14:48.309446+0000","last_fullsized":"2024-02-20T02:14:48.309446+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.303496+0000","last_change":"2024-02-20T02:14:44.224136+0000","last_active":"2024-02-20T02:14:48.303496+0000","last_peered":"2024-02-20T02:14:48.303496+0000","last_clean":"2024-02-20T02:14:48.303496+0000","last_became_active":"2024-02-20T02:14:44.218175+0000","last_became_peered":"2024-02-20T02:14:44.218175+0000","last_unstale":"2024-02-20T02:14:48.303496+0000","last_undegraded":"2024-02-20T02:14:48.303496+0000","last_fullsized":"2024-02-20T02:14:48.303496+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.078198+0000","last_change":"2024-02-20T02:14:44.182566+0000","last_active":"2024-02-20T02:14:47.078198+0000","last_peered":"2024-02-20T02:14:47.078198+0000","last_clean":"2024-02-20T02:14:47.078198+0000","last_became_active":"2024-02-20T02:14:44.175757+0000","last_became_peered":"2024-02-20T02:14:44.175757+0000","last_unstale":"2024-02-20T02:14:47.078198+0000","last_undegraded":"2024-02-20T02:14:47.078198+0000","last_fullsized":"2024-02-20T02:14:47.078198+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.350432+0000","last_change":"2024-02-20T02:14:44.179208+0000","last_active":"2024-02-20T02:14:48.350432+0000","last_peered":"2024-02-20T02:14:48.350432+0000","last_clean":"2024-02-20T02:14:48.350432+0000","last_became_active":"2024-02-20T02:14:44.167479+0000","last_became_peered":"2024-02-20T02:14:44.167479+0000","last_unstale":"2024-02-20T02:14:48.350432+0000","last_undegraded":"2024-02-20T02:14:48.350432+0000","last_fullsized":"2024-02-20T02:14:48.350432+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.338096+0000","last_change":"2024-02-20T02:14:44.192711+0000","last_active":"2024-02-20T02:14:48.338096+0000","last_peered":"2024-02-20T02:14:48.338096+0000","last_clean":"2024-02-20T02:14:48.338096+0000","last_became_active":"2024-02-20T02:14:44.180769+0000","last_became_peered":"2024-02-20T02:14:44.180769+0000","last_unstale":"2024-02-20T02:14:48.338096+0000","last_undegraded":"2024-02-20T02:14:48.338096+0000","last_fullsized":"2024-02-20T02:14:48.338096+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.327481+0000","last_change":"2024-02-20T02:14:44.191580+0000","last_active":"2024-02-20T02:14:48.327481+0000","last_peered":"2024-02-20T02:14:48.327481+0000","last_clean":"2024-02-20T02:14:48.327481+0000","last_became_active":"2024-02-20T02:14:44.182801+0000","last_became_peered":"2024-02-20T02:14:44.182801+0000","last_unstale":"2024-02-20T02:14:48.327481+0000","last_undegraded":"2024-02-20T02:14:48.327481+0000","last_fullsized":"2024-02-20T02:14:48.327481+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.060000+0000","last_change":"2024-02-20T02:14:44.190778+0000","last_active":"2024-02-20T02:14:47.060000+0000","last_peered":"2024-02-20T02:14:47.060000+0000","last_clean":"2024-02-20T02:14:47.060000+0000","last_became_active":"2024-02-20T02:14:44.186481+0000","last_became_peered":"2024-02-20T02:14:44.186481+0000","last_unstale":"2024-02-20T02:14:47.060000+0000","last_undegraded":"2024-02-20T02:14:47.060000+0000","last_fullsized":"2024-02-20T02:14:47.060000+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.259046+0000","last_change":"2024-02-20T02:14:44.246778+0000","last_active":"2024-02-20T02:14:48.259046+0000","last_peered":"2024-02-20T02:14:48.259046+0000","last_clean":"2024-02-20T02:14:48.259046+0000","last_became_active":"2024-02-20T02:14:44.243754+0000","last_became_peered":"2024-02-20T02:14:44.243754+0000","last_unstale":"2024-02-20T02:14:48.259046+0000","last_undegraded":"2024-02-20T02:14:48.259046+0000","last_fullsized":"2024-02-20T02:14:48.259046+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.267425+0000","last_change":"2024-02-20T02:14:44.205105+0000","last_active":"2024-02-20T02:14:48.267425+0000","last_peered":"2024-02-20T02:14:48.267425+0000","last_clean":"2024-02-20T02:14:48.267425+0000","last_became_active":"2024-02-20T02:14:44.194528+0000","last_became_peered":"2024-02-20T02:14:44.194528+0000","last_unstale":"2024-02-20T02:14:48.267425+0000","last_undegraded":"2024-02-20T02:14:48.267425+0000","last_fullsized":"2024-02-20T02:14:48.267425+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.312011+0000","last_change":"2024-02-20T02:14:44.248214+0000","last_active":"2024-02-20T02:14:48.312011+0000","last_peered":"2024-02-20T02:14:48.312011+0000","last_clean":"2024-02-20T02:14:48.312011+0000","last_became_active":"2024-02-20T02:14:44.244704+0000","last_became_peered":"2024-02-20T02:14:44.244704+0000","last_unstale":"2024-02-20T02:14:48.312011+0000","last_undegraded":"2024-02-20T02:14:48.312011+0000","last_fullsized":"2024-02-20T02:14:48.312011+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.295402+0000","last_change":"2024-02-20T02:14:44.238032+0000","last_active":"2024-02-20T02:14:48.295402+0000","last_peered":"2024-02-20T02:14:48.295402+0000","last_clean":"2024-02-20T02:14:48.295402+0000","last_became_active":"2024-02-20T02:14:44.234764+0000","last_became_peered":"2024-02-20T02:14:44.234764+0000","last_unstale":"2024-02-20T02:14:48.295402+0000","last_undegraded":"2024-02-20T02:14:48.295402+0000","last_fullsized":"2024-02-20T02:14:48.295402+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.308248+0000","last_change":"2024-02-20T02:14:44.241835+0000","last_active":"2024-02-20T02:14:48.308248+0000","last_peered":"2024-02-20T02:14:48.308248+0000","last_clean":"2024-02-20T02:14:48.308248+0000","last_became_active":"2024-02-20T02:14:44.238057+0000","last_became_peered":"2024-02-20T02:14:44.238057+0000","last_unstale":"2024-02-20T02:14:48.308248+0000","last_undegraded":"2024-02-20T02:14:48.308248+0000","last_fullsized":"2024-02-20T02:14:48.308248+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.317991+0000","last_change":"2024-02-20T02:14:44.199397+0000","last_active":"2024-02-20T02:14:48.317991+0000","last_peered":"2024-02-20T02:14:48.317991+0000","last_clean":"2024-02-20T02:14:48.317991+0000","last_became_active":"2024-02-20T02:14:44.182848+0000","last_became_peered":"2024-02-20T02:14:44.182848+0000","last_unstale":"2024-02-20T02:14:48.317991+0000","last_undegraded":"2024-02-20T02:14:48.317991+0000","last_fullsized":"2024-02-20T02:14:48.317991+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328213+0000","last_change":"2024-02-20T02:14:44.202362+0000","last_active":"2024-02-20T02:14:48.328213+0000","last_peered":"2024-02-20T02:14:48.328213+0000","last_clean":"2024-02-20T02:14:48.328213+0000","last_became_active":"2024-02-20T02:14:44.191242+0000","last_became_peered":"2024-02-20T02:14:44.191242+0000","last_unstale":"2024-02-20T02:14:48.328213+0000","last_undegraded":"2024-02-20T02:14:48.328213+0000","last_fullsized":"2024-02-20T02:14:48.328213+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"238'7","reported_seq":30,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349794+0000","last_change":"2024-02-20T02:14:44.187038+0000","last_active":"2024-02-20T02:14:48.349794+0000","last_peered":"2024-02-20T02:14:48.349794+0000","last_clean":"2024-02-20T02:14:48.349794+0000","last_became_active":"2024-02-20T02:14:44.172077+0000","last_became_peered":"2024-02-20T02:14:44.172077+0000","last_unstale":"2024-02-20T02:14:48.349794+0000","last_undegraded":"2024-02-20T02:14:48.349794+0000","last_fullsized":"2024-02-20T02:14:48.349794+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.068690+0000","last_change":"2024-02-20T02:14:44.178106+0000","last_active":"2024-02-20T02:14:47.068690+0000","last_peered":"2024-02-20T02:14:47.068690+0000","last_clean":"2024-02-20T02:14:47.068690+0000","last_became_active":"2024-02-20T02:14:44.171906+0000","last_became_peered":"2024-02-20T02:14:44.171906+0000","last_unstale":"2024-02-20T02:14:47.068690+0000","last_undegraded":"2024-02-20T02:14:47.068690+0000","last_fullsized":"2024-02-20T02:14:47.068690+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.351035+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:14:48.351035+0000","last_peered":"2024-02-20T02:14:48.351035+0000","last_clean":"2024-02-20T02:14:48.351035+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:14:48.351035+0000","last_undegraded":"2024-02-20T02:14:48.351035+0000","last_fullsized":"2024-02-20T02:14:48.351035+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.064824+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:14:47.064824+0000","last_peered":"2024-02-20T02:14:47.064824+0000","last_clean":"2024-02-20T02:14:47.064824+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:14:47.064824+0000","last_undegraded":"2024-02-20T02:14:47.064824+0000","last_fullsized":"2024-02-20T02:14:47.064824+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":609,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.299517+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:14:48.299517+0000","last_peered":"2024-02-20T02:14:48.299517+0000","last_clean":"2024-02-20T02:14:48.299517+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:14:48.299517+0000","last_undegraded":"2024-02-20T02:14:48.299517+0000","last_fullsized":"2024-02-20T02:14:48.299517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.071227+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:14:47.071227+0000","last_peered":"2024-02-20T02:14:47.071227+0000","last_clean":"2024-02-20T02:14:47.071227+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:14:47.071227+0000","last_undegraded":"2024-02-20T02:14:47.071227+0000","last_fullsized":"2024-02-20T02:14:47.071227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.289711+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:14:48.289711+0000","last_peered":"2024-02-20T02:14:48.289711+0000","last_clean":"2024-02-20T02:14:48.289711+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:14:48.289711+0000","last_undegraded":"2024-02-20T02:14:48.289711+0000","last_fullsized":"2024-02-20T02:14:48.289711+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349125+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:14:48.349125+0000","last_peered":"2024-02-20T02:14:48.349125+0000","last_clean":"2024-02-20T02:14:48.349125+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:14:48.349125+0000","last_undegraded":"2024-02-20T02:14:48.349125+0000","last_fullsized":"2024-02-20T02:14:48.349125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.325381+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:14:48.325381+0000","last_peered":"2024-02-20T02:14:48.325381+0000","last_clean":"2024-02-20T02:14:48.325381+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:14:48.325381+0000","last_undegraded":"2024-02-20T02:14:48.325381+0000","last_fullsized":"2024-02-20T02:14:48.325381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.367387+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:14:48.367387+0000","last_peered":"2024-02-20T02:14:48.367387+0000","last_clean":"2024-02-20T02:14:48.367387+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:14:48.367387+0000","last_undegraded":"2024-02-20T02:14:48.367387+0000","last_fullsized":"2024-02-20T02:14:48.367387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":745,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.269283+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:14:48.269283+0000","last_peered":"2024-02-20T02:14:48.269283+0000","last_clean":"2024-02-20T02:14:48.269283+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:14:48.269283+0000","last_undegraded":"2024-02-20T02:14:48.269283+0000","last_fullsized":"2024-02-20T02:14:48.269283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":42360,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815240,"statfs":{"total":107374182400,"available":107330805760,"internally_reserved":0,"allocated":2998272,"data_stored":1458322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":37576,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":35264,"kb_avail":104820024,"statfs":{"total":107374182400,"available":107335704576,"internally_reserved":0,"allocated":2359296,"data_stored":846583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36110336},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":41700,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815900,"statfs":{"total":107374182400,"available":107331481600,"internally_reserved":0,"allocated":2322432,"data_stored":845431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":46460,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":43584,"kb_avail":104811140,"statfs":{"total":107374182400,"available":107326607360,"internally_reserved":0,"allocated":2936832,"data_stored":1436562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44630016},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:14:53.350 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-20T02:14:53.351 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:14:53.457 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.455+0000 7f790046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.458 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.459+0000 7f790046d640 1 Processor -- start 2024-02-20T02:14:53.458 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.459+0000 7f790046d640 1 -- start start 2024-02-20T02:14:53.459 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f790046d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f78f8133ab0 0x7f78f8135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f790046d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f8130840 con 0x7f78f8130c20 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f790046d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f8131740 con 0x7f78f8133ab0 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fda08640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f78f8133ab0 0x7f78f8135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fe209640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f78f8130c20 0x7f78f8131030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45560/0 (socket says 172.21.15.125:45560) 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fe209640 1 -- 172.21.15.125:0/1623425756 learned_addr learned my addr 172.21.15.125:0/1623425756 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fda08640 1 -- 172.21.15.125:0/1623425756 >> v1:172.21.15.125:6789/0 conn(0x7f78f8130c20 legacy=0x7f78f8131030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.460 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fd207640 1 -- 172.21.15.125:0/1623425756 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174619955 0 0) 0x7f78f8130840 con 0x7f78f8130c20 2024-02-20T02:14:53.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.460+0000 7f78fda08640 1 -- 172.21.15.125:0/1623425756 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f806f2f0 con 0x7f78f8133ab0 2024-02-20T02:14:53.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f78fda08640 1 --2- 172.21.15.125:0/1623425756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8133ab0 0x7f78f8135ed0 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a85761156016f31 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f78fd207640 1 -- 172.21.15.125:0/1623425756 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78e8009ce0 con 0x7f78f8133ab0 2024-02-20T02:14:53.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f78fd207640 1 -- 172.21.15.125:0/1623425756 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f78e8009e40 con 0x7f78f8133ab0 2024-02-20T02:14:53.461 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f78fd207640 1 -- 172.21.15.125:0/1623425756 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78e800e360 con 0x7f78f8133ab0 2024-02-20T02:14:53.463 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 -- 172.21.15.125:0/1623425756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8133ab0 msgr2=0x7f78f8135ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.464 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 --2- 172.21.15.125:0/1623425756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8133ab0 0x7f78f8135ed0 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.464 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 -- 172.21.15.125:0/1623425756 shutdown_connections 2024-02-20T02:14:53.464 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 --2- 172.21.15.125:0/1623425756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8133ab0 0x7f78f8135ed0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.464 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 -- 172.21.15.125:0/1623425756 >> 172.21.15.125:0/1623425756 conn(0x7f78f809fae0 msgr2=0x7f78f809fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.464 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 -- 172.21.15.125:0/1623425756 shutdown_connections 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.461+0000 7f790046d640 1 -- 172.21.15.125:0/1623425756 wait complete. 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.462+0000 7f790046d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.462+0000 7f790046d640 1 Processor -- start 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f790046d640 1 -- start start 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f790046d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f790046d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f811b090 con 0x7f78f8130c20 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f78fe209640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f78fe209640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39354/0 (socket says 172.21.15.125:39354) 2024-02-20T02:14:53.465 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.463+0000 7f78fe209640 1 -- 172.21.15.125:0/1383127191 learned_addr learned my addr 172.21.15.125:0/1383127191 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f78fe209640 1 -- 172.21.15.125:0/1383127191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f811c650 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f78fe209640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78f4004af0 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f78f4004c50 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f811dd70 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78f4007120 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.464+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f811e6c0 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.465+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f78f40072d0 con 0x7f78f8130c20 2024-02-20T02:14:53.466 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.465+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f811df60 con 0x7f78f8130c20 2024-02-20T02:14:53.467 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.465+0000 7f78eeffd640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 0x7f78dc03c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.467 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.465+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f78f4047700 con 0x7f78f8130c20 2024-02-20T02:14:53.467 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.465+0000 7f78fda08640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 0x7f78dc03c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.467 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.466+0000 7f78fda08640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 0x7f78dc03c600 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.468 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.470+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f78f400a9e0 con 0x7f78f8130c20 2024-02-20T02:14:53.637 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.638+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78f80a17b0 con 0x7f78dc03a0f0 2024-02-20T02:14:53.640 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.642+0000 7f78eeffd640 1 -- 172.21.15.125:0/1383127191 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107066 (crc 0 0 0) 0x7f78e8009cb0 con 0x7f78dc03a0f0 2024-02-20T02:14:53.642 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:14:53.642 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:14:53.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 msgr2=0x7f78dc03c600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 0x7f78dc03c600 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 msgr2=0x7f78f811b750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 shutdown_connections 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f78dc03a0f0 0x7f78dc03c600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 --2- 172.21.15.125:0/1383127191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78f8130c20 0x7f78f811b750 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 >> 172.21.15.125:0/1383127191 conn(0x7f78f809fae0 msgr2=0x7f78f812ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.646 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.643+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 shutdown_connections 2024-02-20T02:14:53.647 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.644+0000 7f790046d640 1 -- 172.21.15.125:0/1383127191 wait complete. 2024-02-20T02:14:53.653 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-02-20T02:14:52.496004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":168096,"kb_used_data":10368,"kb_used_omap":0,"kb_used_meta":157696,"kb_avail":419262304,"statfs":{"total":429496729600,"available":429324599296,"internally_reserved":0,"allocated":10616832,"data_stored":4586898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":161480704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1152,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-24,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001140"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.336365+0000","last_change":"2024-02-20T02:14:44.190037+0000","last_active":"2024-02-20T02:14:48.336365+0000","last_peered":"2024-02-20T02:14:48.336365+0000","last_clean":"2024-02-20T02:14:48.336365+0000","last_became_active":"2024-02-20T02:14:44.170164+0000","last_became_peered":"2024-02-20T02:14:44.170164+0000","last_unstale":"2024-02-20T02:14:48.336365+0000","last_undegraded":"2024-02-20T02:14:48.336365+0000","last_fullsized":"2024-02-20T02:14:48.336365+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.300201+0000","last_change":"2024-02-20T02:14:44.201273+0000","last_active":"2024-02-20T02:14:48.300201+0000","last_peered":"2024-02-20T02:14:48.300201+0000","last_clean":"2024-02-20T02:14:48.300201+0000","last_became_active":"2024-02-20T02:14:44.189022+0000","last_became_peered":"2024-02-20T02:14:44.189022+0000","last_unstale":"2024-02-20T02:14:48.300201+0000","last_undegraded":"2024-02-20T02:14:48.300201+0000","last_fullsized":"2024-02-20T02:14:48.300201+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":18,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.079273+0000","last_change":"2024-02-20T02:14:44.181287+0000","last_active":"2024-02-20T02:14:47.079273+0000","last_peered":"2024-02-20T02:14:47.079273+0000","last_clean":"2024-02-20T02:14:47.079273+0000","last_became_active":"2024-02-20T02:14:44.173755+0000","last_became_peered":"2024-02-20T02:14:44.173755+0000","last_unstale":"2024-02-20T02:14:47.079273+0000","last_undegraded":"2024-02-20T02:14:47.079273+0000","last_fullsized":"2024-02-20T02:14:47.079273+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.280039+0000","last_change":"2024-02-20T02:14:44.211892+0000","last_active":"2024-02-20T02:14:48.280039+0000","last_peered":"2024-02-20T02:14:48.280039+0000","last_clean":"2024-02-20T02:14:48.280039+0000","last_became_active":"2024-02-20T02:14:44.203048+0000","last_became_peered":"2024-02-20T02:14:44.203048+0000","last_unstale":"2024-02-20T02:14:48.280039+0000","last_undegraded":"2024-02-20T02:14:48.280039+0000","last_fullsized":"2024-02-20T02:14:48.280039+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.326420+0000","last_change":"2024-02-20T02:14:44.206069+0000","last_active":"2024-02-20T02:14:48.326420+0000","last_peered":"2024-02-20T02:14:48.326420+0000","last_clean":"2024-02-20T02:14:48.326420+0000","last_became_active":"2024-02-20T02:14:44.201917+0000","last_became_peered":"2024-02-20T02:14:44.201917+0000","last_unstale":"2024-02-20T02:14:48.326420+0000","last_undegraded":"2024-02-20T02:14:48.326420+0000","last_fullsized":"2024-02-20T02:14:48.326420+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292267+0000","last_change":"2024-02-20T02:14:44.187033+0000","last_active":"2024-02-20T02:14:48.292267+0000","last_peered":"2024-02-20T02:14:48.292267+0000","last_clean":"2024-02-20T02:14:48.292267+0000","last_became_active":"2024-02-20T02:14:44.176812+0000","last_became_peered":"2024-02-20T02:14:44.176812+0000","last_unstale":"2024-02-20T02:14:48.292267+0000","last_undegraded":"2024-02-20T02:14:48.292267+0000","last_fullsized":"2024-02-20T02:14:48.292267+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.292992+0000","last_change":"2024-02-20T02:14:44.180791+0000","last_active":"2024-02-20T02:14:48.292992+0000","last_peered":"2024-02-20T02:14:48.292992+0000","last_clean":"2024-02-20T02:14:48.292992+0000","last_became_active":"2024-02-20T02:14:44.173660+0000","last_became_peered":"2024-02-20T02:14:44.173660+0000","last_unstale":"2024-02-20T02:14:48.292992+0000","last_undegraded":"2024-02-20T02:14:48.292992+0000","last_fullsized":"2024-02-20T02:14:48.292992+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328854+0000","last_change":"2024-02-20T02:14:44.204673+0000","last_active":"2024-02-20T02:14:48.328854+0000","last_peered":"2024-02-20T02:14:48.328854+0000","last_clean":"2024-02-20T02:14:48.328854+0000","last_became_active":"2024-02-20T02:14:44.193652+0000","last_became_peered":"2024-02-20T02:14:44.193652+0000","last_unstale":"2024-02-20T02:14:48.328854+0000","last_undegraded":"2024-02-20T02:14:48.328854+0000","last_fullsized":"2024-02-20T02:14:48.328854+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.284071+0000","last_change":"2024-02-20T02:14:44.208695+0000","last_active":"2024-02-20T02:14:48.284071+0000","last_peered":"2024-02-20T02:14:48.284071+0000","last_clean":"2024-02-20T02:14:48.284071+0000","last_became_active":"2024-02-20T02:14:44.199375+0000","last_became_peered":"2024-02-20T02:14:44.199375+0000","last_unstale":"2024-02-20T02:14:48.284071+0000","last_undegraded":"2024-02-20T02:14:48.284071+0000","last_fullsized":"2024-02-20T02:14:48.284071+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.063127+0000","last_change":"2024-02-20T02:14:44.203898+0000","last_active":"2024-02-20T02:14:47.063127+0000","last_peered":"2024-02-20T02:14:47.063127+0000","last_clean":"2024-02-20T02:14:47.063127+0000","last_became_active":"2024-02-20T02:14:44.200473+0000","last_became_peered":"2024-02-20T02:14:44.200473+0000","last_unstale":"2024-02-20T02:14:47.063127+0000","last_undegraded":"2024-02-20T02:14:47.063127+0000","last_fullsized":"2024-02-20T02:14:47.063127+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.298985+0000","last_change":"2024-02-20T02:14:44.196513+0000","last_active":"2024-02-20T02:14:48.298985+0000","last_peered":"2024-02-20T02:14:48.298985+0000","last_clean":"2024-02-20T02:14:48.298985+0000","last_became_active":"2024-02-20T02:14:44.190055+0000","last_became_peered":"2024-02-20T02:14:44.190055+0000","last_unstale":"2024-02-20T02:14:48.298985+0000","last_undegraded":"2024-02-20T02:14:48.298985+0000","last_fullsized":"2024-02-20T02:14:48.298985+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.306517+0000","last_change":"2024-02-20T02:14:44.195328+0000","last_active":"2024-02-20T02:14:48.306517+0000","last_peered":"2024-02-20T02:14:48.306517+0000","last_clean":"2024-02-20T02:14:48.306517+0000","last_became_active":"2024-02-20T02:14:44.184534+0000","last_became_peered":"2024-02-20T02:14:44.184534+0000","last_unstale":"2024-02-20T02:14:48.306517+0000","last_undegraded":"2024-02-20T02:14:48.306517+0000","last_fullsized":"2024-02-20T02:14:48.306517+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.278172+0000","last_change":"2024-02-20T02:14:44.201331+0000","last_active":"2024-02-20T02:14:48.278172+0000","last_peered":"2024-02-20T02:14:48.278172+0000","last_clean":"2024-02-20T02:14:48.278172+0000","last_became_active":"2024-02-20T02:14:44.192136+0000","last_became_peered":"2024-02-20T02:14:44.192136+0000","last_unstale":"2024-02-20T02:14:48.278172+0000","last_undegraded":"2024-02-20T02:14:48.278172+0000","last_fullsized":"2024-02-20T02:14:48.278172+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.271811+0000","last_change":"2024-02-20T02:14:44.258170+0000","last_active":"2024-02-20T02:14:48.271811+0000","last_peered":"2024-02-20T02:14:48.271811+0000","last_clean":"2024-02-20T02:14:48.271811+0000","last_became_active":"2024-02-20T02:14:44.254178+0000","last_became_peered":"2024-02-20T02:14:44.254178+0000","last_unstale":"2024-02-20T02:14:48.271811+0000","last_undegraded":"2024-02-20T02:14:48.271811+0000","last_fullsized":"2024-02-20T02:14:48.271811+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.319527+0000","last_change":"2024-02-20T02:14:44.253515+0000","last_active":"2024-02-20T02:14:48.319527+0000","last_peered":"2024-02-20T02:14:48.319527+0000","last_clean":"2024-02-20T02:14:48.319527+0000","last_became_active":"2024-02-20T02:14:44.250168+0000","last_became_peered":"2024-02-20T02:14:44.250168+0000","last_unstale":"2024-02-20T02:14:48.319527+0000","last_undegraded":"2024-02-20T02:14:48.319527+0000","last_fullsized":"2024-02-20T02:14:48.319527+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.316346+0000","last_change":"2024-02-20T02:14:44.212643+0000","last_active":"2024-02-20T02:14:48.316346+0000","last_peered":"2024-02-20T02:14:48.316346+0000","last_clean":"2024-02-20T02:14:48.316346+0000","last_became_active":"2024-02-20T02:14:44.208652+0000","last_became_peered":"2024-02-20T02:14:44.208652+0000","last_unstale":"2024-02-20T02:14:48.316346+0000","last_undegraded":"2024-02-20T02:14:48.316346+0000","last_fullsized":"2024-02-20T02:14:48.316346+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.309446+0000","last_change":"2024-02-20T02:14:44.206611+0000","last_active":"2024-02-20T02:14:48.309446+0000","last_peered":"2024-02-20T02:14:48.309446+0000","last_clean":"2024-02-20T02:14:48.309446+0000","last_became_active":"2024-02-20T02:14:44.197136+0000","last_became_peered":"2024-02-20T02:14:44.197136+0000","last_unstale":"2024-02-20T02:14:48.309446+0000","last_undegraded":"2024-02-20T02:14:48.309446+0000","last_fullsized":"2024-02-20T02:14:48.309446+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.303496+0000","last_change":"2024-02-20T02:14:44.224136+0000","last_active":"2024-02-20T02:14:48.303496+0000","last_peered":"2024-02-20T02:14:48.303496+0000","last_clean":"2024-02-20T02:14:48.303496+0000","last_became_active":"2024-02-20T02:14:44.218175+0000","last_became_peered":"2024-02-20T02:14:44.218175+0000","last_unstale":"2024-02-20T02:14:48.303496+0000","last_undegraded":"2024-02-20T02:14:48.303496+0000","last_fullsized":"2024-02-20T02:14:48.303496+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.078198+0000","last_change":"2024-02-20T02:14:44.182566+0000","last_active":"2024-02-20T02:14:47.078198+0000","last_peered":"2024-02-20T02:14:47.078198+0000","last_clean":"2024-02-20T02:14:47.078198+0000","last_became_active":"2024-02-20T02:14:44.175757+0000","last_became_peered":"2024-02-20T02:14:44.175757+0000","last_unstale":"2024-02-20T02:14:47.078198+0000","last_undegraded":"2024-02-20T02:14:47.078198+0000","last_fullsized":"2024-02-20T02:14:47.078198+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.350432+0000","last_change":"2024-02-20T02:14:44.179208+0000","last_active":"2024-02-20T02:14:48.350432+0000","last_peered":"2024-02-20T02:14:48.350432+0000","last_clean":"2024-02-20T02:14:48.350432+0000","last_became_active":"2024-02-20T02:14:44.167479+0000","last_became_peered":"2024-02-20T02:14:44.167479+0000","last_unstale":"2024-02-20T02:14:48.350432+0000","last_undegraded":"2024-02-20T02:14:48.350432+0000","last_fullsized":"2024-02-20T02:14:48.350432+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.338096+0000","last_change":"2024-02-20T02:14:44.192711+0000","last_active":"2024-02-20T02:14:48.338096+0000","last_peered":"2024-02-20T02:14:48.338096+0000","last_clean":"2024-02-20T02:14:48.338096+0000","last_became_active":"2024-02-20T02:14:44.180769+0000","last_became_peered":"2024-02-20T02:14:44.180769+0000","last_unstale":"2024-02-20T02:14:48.338096+0000","last_undegraded":"2024-02-20T02:14:48.338096+0000","last_fullsized":"2024-02-20T02:14:48.338096+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.327481+0000","last_change":"2024-02-20T02:14:44.191580+0000","last_active":"2024-02-20T02:14:48.327481+0000","last_peered":"2024-02-20T02:14:48.327481+0000","last_clean":"2024-02-20T02:14:48.327481+0000","last_became_active":"2024-02-20T02:14:44.182801+0000","last_became_peered":"2024-02-20T02:14:44.182801+0000","last_unstale":"2024-02-20T02:14:48.327481+0000","last_undegraded":"2024-02-20T02:14:48.327481+0000","last_fullsized":"2024-02-20T02:14:48.327481+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.060000+0000","last_change":"2024-02-20T02:14:44.190778+0000","last_active":"2024-02-20T02:14:47.060000+0000","last_peered":"2024-02-20T02:14:47.060000+0000","last_clean":"2024-02-20T02:14:47.060000+0000","last_became_active":"2024-02-20T02:14:44.186481+0000","last_became_peered":"2024-02-20T02:14:44.186481+0000","last_unstale":"2024-02-20T02:14:47.060000+0000","last_undegraded":"2024-02-20T02:14:47.060000+0000","last_fullsized":"2024-02-20T02:14:47.060000+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.259046+0000","last_change":"2024-02-20T02:14:44.246778+0000","last_active":"2024-02-20T02:14:48.259046+0000","last_peered":"2024-02-20T02:14:48.259046+0000","last_clean":"2024-02-20T02:14:48.259046+0000","last_became_active":"2024-02-20T02:14:44.243754+0000","last_became_peered":"2024-02-20T02:14:44.243754+0000","last_unstale":"2024-02-20T02:14:48.259046+0000","last_undegraded":"2024-02-20T02:14:48.259046+0000","last_fullsized":"2024-02-20T02:14:48.259046+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.267425+0000","last_change":"2024-02-20T02:14:44.205105+0000","last_active":"2024-02-20T02:14:48.267425+0000","last_peered":"2024-02-20T02:14:48.267425+0000","last_clean":"2024-02-20T02:14:48.267425+0000","last_became_active":"2024-02-20T02:14:44.194528+0000","last_became_peered":"2024-02-20T02:14:44.194528+0000","last_unstale":"2024-02-20T02:14:48.267425+0000","last_undegraded":"2024-02-20T02:14:48.267425+0000","last_fullsized":"2024-02-20T02:14:48.267425+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.312011+0000","last_change":"2024-02-20T02:14:44.248214+0000","last_active":"2024-02-20T02:14:48.312011+0000","last_peered":"2024-02-20T02:14:48.312011+0000","last_clean":"2024-02-20T02:14:48.312011+0000","last_became_active":"2024-02-20T02:14:44.244704+0000","last_became_peered":"2024-02-20T02:14:44.244704+0000","last_unstale":"2024-02-20T02:14:48.312011+0000","last_undegraded":"2024-02-20T02:14:48.312011+0000","last_fullsized":"2024-02-20T02:14:48.312011+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.295402+0000","last_change":"2024-02-20T02:14:44.238032+0000","last_active":"2024-02-20T02:14:48.295402+0000","last_peered":"2024-02-20T02:14:48.295402+0000","last_clean":"2024-02-20T02:14:48.295402+0000","last_became_active":"2024-02-20T02:14:44.234764+0000","last_became_peered":"2024-02-20T02:14:44.234764+0000","last_unstale":"2024-02-20T02:14:48.295402+0000","last_undegraded":"2024-02-20T02:14:48.295402+0000","last_fullsized":"2024-02-20T02:14:48.295402+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.308248+0000","last_change":"2024-02-20T02:14:44.241835+0000","last_active":"2024-02-20T02:14:48.308248+0000","last_peered":"2024-02-20T02:14:48.308248+0000","last_clean":"2024-02-20T02:14:48.308248+0000","last_became_active":"2024-02-20T02:14:44.238057+0000","last_became_peered":"2024-02-20T02:14:44.238057+0000","last_unstale":"2024-02-20T02:14:48.308248+0000","last_undegraded":"2024-02-20T02:14:48.308248+0000","last_fullsized":"2024-02-20T02:14:48.308248+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.317991+0000","last_change":"2024-02-20T02:14:44.199397+0000","last_active":"2024-02-20T02:14:48.317991+0000","last_peered":"2024-02-20T02:14:48.317991+0000","last_clean":"2024-02-20T02:14:48.317991+0000","last_became_active":"2024-02-20T02:14:44.182848+0000","last_became_peered":"2024-02-20T02:14:44.182848+0000","last_unstale":"2024-02-20T02:14:48.317991+0000","last_undegraded":"2024-02-20T02:14:48.317991+0000","last_fullsized":"2024-02-20T02:14:48.317991+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":17,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.328213+0000","last_change":"2024-02-20T02:14:44.202362+0000","last_active":"2024-02-20T02:14:48.328213+0000","last_peered":"2024-02-20T02:14:48.328213+0000","last_clean":"2024-02-20T02:14:48.328213+0000","last_became_active":"2024-02-20T02:14:44.191242+0000","last_became_peered":"2024-02-20T02:14:44.191242+0000","last_unstale":"2024-02-20T02:14:48.328213+0000","last_undegraded":"2024-02-20T02:14:48.328213+0000","last_fullsized":"2024-02-20T02:14:48.328213+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"238'7","reported_seq":30,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349794+0000","last_change":"2024-02-20T02:14:44.187038+0000","last_active":"2024-02-20T02:14:48.349794+0000","last_peered":"2024-02-20T02:14:48.349794+0000","last_clean":"2024-02-20T02:14:48.349794+0000","last_became_active":"2024-02-20T02:14:44.172077+0000","last_became_peered":"2024-02-20T02:14:44.172077+0000","last_unstale":"2024-02-20T02:14:48.349794+0000","last_undegraded":"2024-02-20T02:14:48.349794+0000","last_fullsized":"2024-02-20T02:14:48.349794+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":17,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.068690+0000","last_change":"2024-02-20T02:14:44.178106+0000","last_active":"2024-02-20T02:14:47.068690+0000","last_peered":"2024-02-20T02:14:47.068690+0000","last_clean":"2024-02-20T02:14:47.068690+0000","last_became_active":"2024-02-20T02:14:44.171906+0000","last_became_peered":"2024-02-20T02:14:44.171906+0000","last_unstale":"2024-02-20T02:14:47.068690+0000","last_undegraded":"2024-02-20T02:14:47.068690+0000","last_fullsized":"2024-02-20T02:14:47.068690+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.351035+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:14:48.351035+0000","last_peered":"2024-02-20T02:14:48.351035+0000","last_clean":"2024-02-20T02:14:48.351035+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:14:48.351035+0000","last_undegraded":"2024-02-20T02:14:48.351035+0000","last_fullsized":"2024-02-20T02:14:48.351035+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.064824+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:14:47.064824+0000","last_peered":"2024-02-20T02:14:47.064824+0000","last_clean":"2024-02-20T02:14:47.064824+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:14:47.064824+0000","last_undegraded":"2024-02-20T02:14:47.064824+0000","last_fullsized":"2024-02-20T02:14:47.064824+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":609,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.299517+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:14:48.299517+0000","last_peered":"2024-02-20T02:14:48.299517+0000","last_clean":"2024-02-20T02:14:48.299517+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:14:48.299517+0000","last_undegraded":"2024-02-20T02:14:48.299517+0000","last_fullsized":"2024-02-20T02:14:48.299517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":597,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:47.071227+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:14:47.071227+0000","last_peered":"2024-02-20T02:14:47.071227+0000","last_clean":"2024-02-20T02:14:47.071227+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:14:47.071227+0000","last_undegraded":"2024-02-20T02:14:47.071227+0000","last_fullsized":"2024-02-20T02:14:47.071227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.289711+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:14:48.289711+0000","last_peered":"2024-02-20T02:14:48.289711+0000","last_clean":"2024-02-20T02:14:48.289711+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:14:48.289711+0000","last_undegraded":"2024-02-20T02:14:48.289711+0000","last_fullsized":"2024-02-20T02:14:48.289711+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.349125+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:14:48.349125+0000","last_peered":"2024-02-20T02:14:48.349125+0000","last_clean":"2024-02-20T02:14:48.349125+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:14:48.349125+0000","last_undegraded":"2024-02-20T02:14:48.349125+0000","last_fullsized":"2024-02-20T02:14:48.349125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.325381+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:14:48.325381+0000","last_peered":"2024-02-20T02:14:48.325381+0000","last_clean":"2024-02-20T02:14:48.325381+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:14:48.325381+0000","last_undegraded":"2024-02-20T02:14:48.325381+0000","last_fullsized":"2024-02-20T02:14:48.325381+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":603,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.367387+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:14:48.367387+0000","last_peered":"2024-02-20T02:14:48.367387+0000","last_clean":"2024-02-20T02:14:48.367387+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:14:48.367387+0000","last_undegraded":"2024-02-20T02:14:48.367387+0000","last_fullsized":"2024-02-20T02:14:48.367387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":745,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-02-20T02:14:48.269283+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:14:48.269283+0000","last_peered":"2024-02-20T02:14:48.269283+0000","last_clean":"2024-02-20T02:14:48.269283+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:14:48.269283+0000","last_undegraded":"2024-02-20T02:14:48.269283+0000","last_fullsized":"2024-02-20T02:14:48.269283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":42360,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815240,"statfs":{"total":107374182400,"available":107330805760,"internally_reserved":0,"allocated":2998272,"data_stored":1458322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":37576,"kb_used_data":2304,"kb_used_omap":0,"kb_used_meta":35264,"kb_avail":104820024,"statfs":{"total":107374182400,"available":107335704576,"internally_reserved":0,"allocated":2359296,"data_stored":846583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36110336},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":41700,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815900,"statfs":{"total":107374182400,"available":107331481600,"internally_reserved":0,"allocated":2322432,"data_stored":845431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":46460,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":43584,"kb_avail":104811140,"statfs":{"total":107374182400,"available":107326607360,"internally_reserved":0,"allocated":2936832,"data_stored":1436562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44630016},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:14:53.655 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-20T02:14:53.761 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.759+0000 7f0c4726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.761 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.763+0000 7f0c4726d640 1 Processor -- start 2024-02-20T02:14:53.761 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.763+0000 7f0c4726d640 1 -- start start 2024-02-20T02:14:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.763+0000 7f0c4726d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0c401342a0 0x7f0c401346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.763+0000 7f0c4726d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4009a810 con 0x7f0c400a17d0 2024-02-20T02:14:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.763+0000 7f0c4726d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c40099f20 con 0x7f0c401342a0 2024-02-20T02:14:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c45009640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0c401342a0 0x7f0c401346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.763 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c44808640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f0c400a17d0 0x7f0c400a1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45576/0 (socket says 172.21.15.125:45576) 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c44808640 1 -- 172.21.15.125:0/2382203933 learned_addr learned my addr 172.21.15.125:0/2382203933 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c45009640 1 -- 172.21.15.125:0/2382203933 >> v1:172.21.15.125:6789/0 conn(0x7f0c400a17d0 legacy=0x7f0c400a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c45009640 1 -- 172.21.15.125:0/2382203933 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4009bbe0 con 0x7f0c401342a0 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c45009640 1 --2- 172.21.15.125:0/2382203933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c401342a0 0x7f0c401346b0 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b2ce04ef1f78c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c37fff640 1 -- 172.21.15.125:0/2382203933 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c3c009cd0 con 0x7f0c401342a0 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c37fff640 1 -- 172.21.15.125:0/2382203933 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c3c009e30 con 0x7f0c401342a0 2024-02-20T02:14:53.764 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.764+0000 7f0c37fff640 1 -- 172.21.15.125:0/2382203933 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c3c00e360 con 0x7f0c401342a0 2024-02-20T02:14:53.766 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 -- 172.21.15.125:0/2382203933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c401342a0 msgr2=0x7f0c401346b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.766 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 --2- 172.21.15.125:0/2382203933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c401342a0 0x7f0c401346b0 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 -- 172.21.15.125:0/2382203933 shutdown_connections 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 --2- 172.21.15.125:0/2382203933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c401342a0 0x7f0c401346b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 -- 172.21.15.125:0/2382203933 >> 172.21.15.125:0/2382203933 conn(0x7f0c4006c8a0 msgr2=0x7f0c4006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 -- 172.21.15.125:0/2382203933 shutdown_connections 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 1 -- 172.21.15.125:0/2382203933 wait complete. 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.765+0000 7f0c4726d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.766+0000 7f0c4726d640 1 Processor -- start 2024-02-20T02:14:53.767 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.766+0000 7f0c4726d640 1 -- start start 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c4726d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c4726d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40130fa0 con 0x7f0c400a17d0 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c45009640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c45009640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39360/0 (socket says 172.21.15.125:39360) 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c45009640 1 -- 172.21.15.125:0/4283500846 learned_addr learned my addr 172.21.15.125:0/4283500846 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c45009640 1 -- 172.21.15.125:0/4283500846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40121070 con 0x7f0c400a17d0 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c45009640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c3c015550 con 0x7f0c400a17d0 2024-02-20T02:14:53.768 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.767+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c3c0156b0 con 0x7f0c400a17d0 2024-02-20T02:14:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.768+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c40120b60 con 0x7f0c400a17d0 2024-02-20T02:14:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.768+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c3c015950 con 0x7f0c400a17d0 2024-02-20T02:14:53.769 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.768+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4011fac0 con 0x7f0c400a17d0 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.769+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f0c3c015b00 con 0x7f0c400a17d0 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.769+0000 7f0c35ffb640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 0x7f0c2403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.769+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f0c3c0490a0 con 0x7f0c400a17d0 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.769+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4011fd10 con 0x7f0c400a17d0 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.769+0000 7f0c44808640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 0x7f0c2403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:53.770 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.770+0000 7f0c44808640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 0x7f0c2403a3f0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:53.772 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.774+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f0c3c014cb0 con 0x7f0c400a17d0 2024-02-20T02:14:53.947 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.948+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c40062a30 con 0x7f0c400a17d0 2024-02-20T02:14:53.948 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.949+0000 7f0c35ffb640 1 -- 172.21.15.125:0/4283500846 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+10547 (crc 0 0 0) 0x7f0c3c010ba0 con 0x7f0c400a17d0 2024-02-20T02:14:53.948 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:14:53.948 INFO:teuthology.orchestra.run.smithi125.stdout:{"epoch":245,"fsid":"7c271077-0df8-4ad5-a525-6f111846375d","created":"2024-02-20T02:09:32.004176+0000","modified":"2024-02-20T02:14:52.046545+0000","last_up_change":"2024-02-20T02:09:39.134569+0000","last_in_change":"2024-02-20T02:09:34.112255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":77,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-20T02:09:38.470855+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-02-20T02:09:42.206626+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-02-20T02:14:42.064998+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"38094573-2a02-424b-9ba5-a0e77fed1db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":243,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":3754305691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6801","nonce":3754305691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6809","nonce":3754305691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":3754305691}]},"public_addr":"172.21.15.125:6802/3754305691","cluster_addr":"172.21.15.125:6801/3754305691","heartbeat_back_addr":"172.21.15.125:6809/3754305691","heartbeat_front_addr":"172.21.15.125:6810/3754305691","state":["exists","up"]},{"osd":1,"uuid":"6b22849a-2204-4970-b546-cd235cfec47c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":243,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":3877308034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6805","nonce":3877308034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":3877308034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6803","nonce":3877308034}]},"public_addr":"172.21.15.125:6804/3877308034","cluster_addr":"172.21.15.125:6805/3877308034","heartbeat_back_addr":"172.21.15.125:6806/3877308034","heartbeat_front_addr":"172.21.15.125:6803/3877308034","state":["exists","up"]},{"osd":2,"uuid":"cc53d5a4-28f8-4fbd-a10a-d0293de49179","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":243,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":774159808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":774159808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":774159808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":774159808}]},"public_addr":"172.21.15.204:6811/774159808","cluster_addr":"172.21.15.204:6812/774159808","heartbeat_back_addr":"172.21.15.204:6808/774159808","heartbeat_front_addr":"172.21.15.204:6807/774159808","state":["exists","up"]},{"osd":3,"uuid":"523402ba-0344-4708-bb63-297d11afad8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":243,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":409650538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":409650538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":409650538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":409650538}]},"public_addr":"172.21.15.204:6804/409650538","cluster_addr":"172.21.15.204:6803/409650538","heartbeat_back_addr":"172.21.15.204:6810/409650538","heartbeat_front_addr":"172.21.15.204:6809/409650538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-20T02:14:53.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 msgr2=0x7f0c2403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.950 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 0x7f0c2403a3f0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 msgr2=0x7f0c4011d460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 shutdown_connections 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0c24037ee0 0x7f0c2403a3f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 --2- 172.21.15.125:0/4283500846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c400a17d0 0x7f0c4011d460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 >> 172.21.15.125:0/4283500846 conn(0x7f0c4006c8a0 msgr2=0x7f0c400a3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 shutdown_connections 2024-02-20T02:14:53.951 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:53.951+0000 7f0c4726d640 1 -- 172.21.15.125:0/4283500846 wait complete. 2024-02-20T02:14:54.962 INFO:tasks.ceph:Scrubbing osd.0 2024-02-20T02:14:54.963 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-02-20T02:14:55.088 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.082+0000 7f5474f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.089 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.090+0000 7f5474f9d640 1 Processor -- start 2024-02-20T02:14:55.089 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.090+0000 7f5474f9d640 1 -- start start 2024-02-20T02:14:55.090 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f5474f9d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f547009bb40 0x7f5470131d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.090 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f5474f9d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700703e0 con 0x7f5470132240 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f5474f9d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5470131820 con 0x7f547009bb40 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546e59c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f547009bb40 0x7f5470131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f5467fff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f5470132240 0x7f5470099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45588/0 (socket says 172.21.15.125:45588) 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f5467fff640 1 -- 172.21.15.125:0/348997786 learned_addr learned my addr 172.21.15.125:0/348997786 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546e59c640 1 -- 172.21.15.125:0/348997786 >> v1:172.21.15.125:6789/0 conn(0x7f5470132240 legacy=0x7f5470099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546e59c640 1 -- 172.21.15.125:0/348997786 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547009b440 con 0x7f547009bb40 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546e59c640 1 --2- 172.21.15.125:0/348997786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f5470131d00 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6714917bd265f22d server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.091 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546dd9b640 1 -- 172.21.15.125:0/348997786 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5460009ce0 con 0x7f547009bb40 2024-02-20T02:14:55.092 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546dd9b640 1 -- 172.21.15.125:0/348997786 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5460009e40 con 0x7f547009bb40 2024-02-20T02:14:55.092 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.091+0000 7f546dd9b640 1 -- 172.21.15.125:0/348997786 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f546000e360 con 0x7f547009bb40 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 -- 172.21.15.125:0/348997786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 msgr2=0x7f5470131d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 --2- 172.21.15.125:0/348997786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f5470131d00 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 -- 172.21.15.125:0/348997786 shutdown_connections 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 --2- 172.21.15.125:0/348997786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f5470131d00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 -- 172.21.15.125:0/348997786 >> 172.21.15.125:0/348997786 conn(0x7f547009fae0 msgr2=0x7f547009fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.094 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 -- 172.21.15.125:0/348997786 shutdown_connections 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 1 -- 172.21.15.125:0/348997786 wait complete. 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.092+0000 7f5474f9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.093+0000 7f5474f9d640 1 Processor -- start 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.093+0000 7f5474f9d640 1 -- start start 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.094+0000 7f5474f9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.094+0000 7f5474f9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547012ef10 con 0x7f547009bb40 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.094+0000 7f546e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.095 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.094+0000 7f546e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39388/0 (socket says 172.21.15.125:39388) 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.094+0000 7f546e59c640 1 -- 172.21.15.125:0/2200467693 learned_addr learned my addr 172.21.15.125:0/2200467693 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f546e59c640 1 -- 172.21.15.125:0/2200467693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547011c020 con 0x7f547009bb40 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f546e59c640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5460014620 con 0x7f547009bb40 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5460014780 con 0x7f547009bb40 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547011d3d0 con 0x7f547009bb40 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5460014a20 con 0x7f547009bb40 2024-02-20T02:14:55.096 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.095+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547011e360 con 0x7f547009bb40 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f547011f100 con 0x7f547009bb40 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f5460014b80 con 0x7f547009bb40 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 0x7f545003a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f5460019040 con 0x7f547009bb40 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 --2- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 0x7f54500400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.098 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 --> v2:172.21.15.125:6802/3754305691 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5450040740 con 0x7f545003dc20 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=245) v2 ==== 24+0+0 (crc 0 0 0) 0x7f5460048dc0 con 0x7f547009bb40 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.096+0000 7f5467fff640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 0x7f545003a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.097+0000 7f546ed9d640 1 --2- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 0x7f54500400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.097+0000 7f5467fff640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 0x7f545003a3f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.098+0000 7f546ed9d640 1 --2- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 0x7f54500400b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.099 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.099+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== osd.0 v2:172.21.15.125:6802/3754305691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f5450040740 con 0x7f545003dc20 2024-02-20T02:14:55.101 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.102+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 --> v2:172.21.15.125:6802/3754305691 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f547009add0 con 0x7f545003dc20 2024-02-20T02:14:55.102 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5466ffd640 1 -- 172.21.15.125:0/2200467693 <== osd.0 v2:172.21.15.125:6802/3754305691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7f546800a820 con 0x7f545003dc20 2024-02-20T02:14:55.102 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-20T02:14:55.102 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "" 2024-02-20T02:14:55.102 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-20T02:14:55.103 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 msgr2=0x7f54500400b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.103 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 0x7f54500400b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.103 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 msgr2=0x7f545003a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 0x7f545003a3f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 msgr2=0x7f547011aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 shutdown_connections 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> v2:172.21.15.125:6802/3754305691 conn(0x7f545003dc20 0x7f54500400b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5450037ee0 0x7f545003a3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 --2- 172.21.15.125:0/2200467693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f547009bb40 0x7f547011aee0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.104 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 >> 172.21.15.125:0/2200467693 conn(0x7f547009fae0 msgr2=0x7f54701b9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.105 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 shutdown_connections 2024-02-20T02:14:55.105 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.103+0000 7f5474f9d640 1 -- 172.21.15.125:0/2200467693 wait complete. 2024-02-20T02:14:55.113 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-02-20T02:14:55.221 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.218+0000 7f01fde6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.222 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.223+0000 7f01fde6d640 1 Processor -- start 2024-02-20T02:14:55.222 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.223+0000 7f01fde6d640 1 -- start start 2024-02-20T02:14:55.223 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01fde6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f01f8133ab0 0x7f01f8135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01fde6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f8130840 con 0x7f01f8130c20 2024-02-20T02:14:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01fde6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f8131740 con 0x7f01f8133ab0 2024-02-20T02:14:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f6ffd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f01f8133ab0 0x7f01f8135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f77fe640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f01f8130c20 0x7f01f8131030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45598/0 (socket says 172.21.15.125:45598) 2024-02-20T02:14:55.224 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f77fe640 1 -- 172.21.15.125:0/1279924332 learned_addr learned my addr 172.21.15.125:0/1279924332 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f6ffd640 1 -- 172.21.15.125:0/1279924332 >> v1:172.21.15.125:6789/0 conn(0x7f01f8130c20 legacy=0x7f01f8131030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f6ffd640 1 -- 172.21.15.125:0/1279924332 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f806f2f0 con 0x7f01f8133ab0 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.224+0000 7f01f6ffd640 1 --2- 172.21.15.125:0/1279924332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8133ab0 0x7f01f8135ed0 crc :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6689d40c2bf6ca server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01f67fc640 1 -- 172.21.15.125:0/1279924332 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01ec009ce0 con 0x7f01f8133ab0 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01f67fc640 1 -- 172.21.15.125:0/1279924332 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f01ec009e40 con 0x7f01f8133ab0 2024-02-20T02:14:55.225 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01f67fc640 1 -- 172.21.15.125:0/1279924332 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01ec00e360 con 0x7f01f8133ab0 2024-02-20T02:14:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 -- 172.21.15.125:0/1279924332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8133ab0 msgr2=0x7f01f8135ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 --2- 172.21.15.125:0/1279924332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8133ab0 0x7f01f8135ed0 crc :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.228 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 -- 172.21.15.125:0/1279924332 shutdown_connections 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 --2- 172.21.15.125:0/1279924332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8133ab0 0x7f01f8135ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 -- 172.21.15.125:0/1279924332 >> 172.21.15.125:0/1279924332 conn(0x7f01f809fae0 msgr2=0x7f01f809fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 -- 172.21.15.125:0/1279924332 shutdown_connections 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.225+0000 7f01fde6d640 1 -- 172.21.15.125:0/1279924332 wait complete. 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.226+0000 7f01fde6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.226+0000 7f01fde6d640 1 Processor -- start 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01fde6d640 1 -- start start 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01fde6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01fde6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f812e1d0 con 0x7f01f8130c20 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39408/0 (socket says 172.21.15.125:39408) 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.227+0000 7f01f77fe640 1 -- 172.21.15.125:0/3768644908 learned_addr learned my addr 172.21.15.125:0/3768644908 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01f77fe640 1 -- 172.21.15.125:0/3768644908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f811c5b0 con 0x7f01f8130c20 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01f77fe640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 crc :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01e80059d0 con 0x7f01f8130c20 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f01e8005b30 con 0x7f01f8130c20 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f811c870 con 0x7f01f8130c20 2024-02-20T02:14:55.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01e8005dd0 con 0x7f01f8130c20 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.228+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f811c120 con 0x7f01f8130c20 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.229+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f01e80070b0 con 0x7f01f8130c20 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.229+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f8062a30 con 0x7f01f8130c20 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.229+0000 7f01dffff640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 0x7f01e403a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.229+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f01e8047520 con 0x7f01f8130c20 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.229+0000 7f01f6ffd640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 0x7f01e403a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.230+0000 7f01f6ffd640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 0x7f01e403a3a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.234+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f01e800d330 con 0x7f01f8130c20 2024-02-20T02:14:55.404 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.405+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f01f81222a0 con 0x7f01e4037e90 2024-02-20T02:14:55.405 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.407+0000 7f01dffff640 1 -- 172.21.15.125:0/3768644908 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f01f81222a0 con 0x7f01e4037e90 2024-02-20T02:14:55.405 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 0 to deep-scrub 2024-02-20T02:14:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 msgr2=0x7f01e403a3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 0x7f01e403a3a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 msgr2=0x7f01f811ade0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.407 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 crc :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 shutdown_connections 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f01e4037e90 0x7f01e403a3a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 --2- 172.21.15.125:0/3768644908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01f8130c20 0x7f01f811ade0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 >> 172.21.15.125:0/3768644908 conn(0x7f01f809fae0 msgr2=0x7f01f812ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 shutdown_connections 2024-02-20T02:14:55.408 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.408+0000 7f01fde6d640 1 -- 172.21.15.125:0/3768644908 wait complete. 2024-02-20T02:14:55.419 INFO:tasks.ceph:Scrubbing osd.1 2024-02-20T02:14:55.420 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-02-20T02:14:55.528 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.522+0000 7ffbbb39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.530+0000 7ffbbb39d640 1 Processor -- start 2024-02-20T02:14:55.529 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.530+0000 7ffbbb39d640 1 -- start start 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbbb39d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7ffbb4130470 0x7ffbb4130880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbbb39d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb409bf80 con 0x7ffbb409a300 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbbb39d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb412fec0 con 0x7ffbb4130470 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbb9139640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7ffbb4130470 0x7ffbb4130880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbb8938640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7ffbb409a300 0x7ffbb409a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45608/0 (socket says 172.21.15.125:45608) 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbb8938640 1 -- 172.21.15.125:0/2552249201 learned_addr learned my addr 172.21.15.125:0/2552249201 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:55.531 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbb9139640 1 -- 172.21.15.125:0/2552249201 >> v1:172.21.15.125:6789/0 conn(0x7ffbb409a300 legacy=0x7ffbb409a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.532 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.531+0000 7ffbb9139640 1 -- 172.21.15.125:0/2552249201 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4130dc0 con 0x7ffbb4130470 2024-02-20T02:14:55.532 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbb9139640 1 --2- 172.21.15.125:0/2552249201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4130470 0x7ffbb4130880 crc :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4985d7263dc9f4ff server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.532 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbabfff640 1 -- 172.21.15.125:0/2552249201 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffbac00f040 con 0x7ffbb4130470 2024-02-20T02:14:55.532 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbabfff640 1 -- 172.21.15.125:0/2552249201 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffbac00a1e0 con 0x7ffbb4130470 2024-02-20T02:14:55.532 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbabfff640 1 -- 172.21.15.125:0/2552249201 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffbac008780 con 0x7ffbb4130470 2024-02-20T02:14:55.535 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbbb39d640 1 -- 172.21.15.125:0/2552249201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4130470 msgr2=0x7ffbb4130880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/2552249201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4130470 0x7ffbb4130880 crc :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbbb39d640 1 -- 172.21.15.125:0/2552249201 shutdown_connections 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/2552249201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4130470 0x7ffbb4130880 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.532+0000 7ffbbb39d640 1 -- 172.21.15.125:0/2552249201 >> 172.21.15.125:0/2552249201 conn(0x7ffbb409fae0 msgr2=0x7ffbb409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.533+0000 7ffbbb39d640 1 -- 172.21.15.125:0/2552249201 shutdown_connections 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.533+0000 7ffbbb39d640 1 -- 172.21.15.125:0/2552249201 wait complete. 2024-02-20T02:14:55.536 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.533+0000 7ffbbb39d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbbb39d640 1 Processor -- start 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbbb39d640 1 -- start start 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbbb39d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbbb39d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb411fa30 con 0x7ffbb409a300 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbb9139640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbb9139640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39422/0 (socket says 172.21.15.125:39422) 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.534+0000 7ffbb9139640 1 -- 172.21.15.125:0/1142380943 learned_addr learned my addr 172.21.15.125:0/1142380943 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:55.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffbb9139640 1 -- 172.21.15.125:0/1142380943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb411fc60 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffbb9139640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffbac00f040 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffbac0107f0 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb4119d50 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffbac010a90 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.535+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb41203f0 con 0x7ffbb409a300 2024-02-20T02:14:55.538 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7ffbac012040 con 0x7ffbb409a300 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffbb41211a0 con 0x7ffbb409a300 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 0x7ffb9803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7ffbac0488f0 con 0x7ffbb409a300 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 --2- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 0x7ffb980400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 --> v2:172.21.15.125:6804/3877308034 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb98040740 con 0x7ffb9803dc20 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=245) v2 ==== 24+0+0 (crc 0 0 0) 0x7ffbac048c40 con 0x7ffbb409a300 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.536+0000 7ffbb8938640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 0x7ffb9803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.537+0000 7ffbb993a640 1 --2- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 0x7ffb980400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:14:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.537+0000 7ffbb8938640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 0x7ffb9803a3f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.538+0000 7ffbb993a640 1 --2- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 0x7ffb980400b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.539+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== osd.1 v2:172.21.15.125:6804/3877308034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7ffb98040740 con 0x7ffb9803dc20 2024-02-20T02:14:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.542+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 --> v2:172.21.15.125:6804/3877308034 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffbb412f4b0 con 0x7ffb9803dc20 2024-02-20T02:14:55.541 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffba9ffb640 1 -- 172.21.15.125:0/1142380943 <== osd.1 v2:172.21.15.125:6804/3877308034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7ffbb412f4b0 con 0x7ffb9803dc20 2024-02-20T02:14:55.542 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-20T02:14:55.542 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "" 2024-02-20T02:14:55.542 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-20T02:14:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 msgr2=0x7ffb980400b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.543 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 0x7ffb980400b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 msgr2=0x7ffb9803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 0x7ffb9803a3f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 msgr2=0x7ffbb4118c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 shutdown_connections 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> v2:172.21.15.125:6804/3877308034 conn(0x7ffb9803dc20 0x7ffb980400b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7ffb98037ee0 0x7ffb9803a3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 --2- 172.21.15.125:0/1142380943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb409a300 0x7ffbb4118c30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 >> 172.21.15.125:0/1142380943 conn(0x7ffbb409fae0 msgr2=0x7ffbb409cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 shutdown_connections 2024-02-20T02:14:55.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.543+0000 7ffbbb39d640 1 -- 172.21.15.125:0/1142380943 wait complete. 2024-02-20T02:14:55.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-02-20T02:14:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.655+0000 7f0ce7a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.660+0000 7f0ce7a6d640 1 Processor -- start 2024-02-20T02:14:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.660+0000 7f0ce7a6d640 1 -- start start 2024-02-20T02:14:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce7a6d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0ce009bea0 0x7f0ce012cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.662 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce7a6d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce009b710 con 0x7f0ce012d2d0 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce7a6d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce006eb50 con 0x7f0ce009bea0 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce5809640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f0ce009bea0 0x7f0ce012cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce5008640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f0ce012d2d0 0x7f0ce012ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45620/0 (socket says 172.21.15.125:45620) 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce5008640 1 -- 172.21.15.125:0/1813994402 learned_addr learned my addr 172.21.15.125:0/1813994402 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce5809640 1 -- 172.21.15.125:0/1813994402 >> v1:172.21.15.125:6789/0 conn(0x7f0ce012d2d0 legacy=0x7f0ce012ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce5809640 1 -- 172.21.15.125:0/1813994402 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce006e070 con 0x7f0ce009bea0 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.661+0000 7f0ce4807640 1 -- 172.21.15.125:0/1813994402 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995305481 0 0) 0x7f0ce009b710 con 0x7f0ce012d2d0 2024-02-20T02:14:55.663 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce5809640 1 --2- 172.21.15.125:0/1813994402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce012cd90 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf9973d588bf8b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce4807640 1 -- 172.21.15.125:0/1813994402 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0cdc009ce0 con 0x7f0ce009bea0 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce4807640 1 -- 172.21.15.125:0/1813994402 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0cdc009e40 con 0x7f0ce009bea0 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce4807640 1 -- 172.21.15.125:0/1813994402 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0cdc00e360 con 0x7f0ce009bea0 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/1813994402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 msgr2=0x7f0ce012cd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/1813994402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce012cd90 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/1813994402 shutdown_connections 2024-02-20T02:14:55.664 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/1813994402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce012cd90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.662+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/1813994402 >> 172.21.15.125:0/1813994402 conn(0x7f0ce009fae0 msgr2=0x7f0ce009fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.663+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/1813994402 shutdown_connections 2024-02-20T02:14:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.663+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/1813994402 wait complete. 2024-02-20T02:14:55.665 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.663+0000 7f0ce7a6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.667 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce7a6d640 1 Processor -- start 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce7a6d640 1 -- start start 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce7a6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce7a6d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01346b0 con 0x7f0ce009bea0 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce5809640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce5809640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39454/0 (socket says 172.21.15.125:39454) 2024-02-20T02:14:55.668 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.664+0000 7f0ce5809640 1 -- 172.21.15.125:0/631272949 learned_addr learned my addr 172.21.15.125:0/631272949 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:55.669 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0ce5809640 1 -- 172.21.15.125:0/631272949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0134410 con 0x7f0ce009bea0 2024-02-20T02:14:55.669 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0ce5809640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.671 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0cdc014410 con 0x7f0ce009bea0 2024-02-20T02:14:55.671 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0cdc014570 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce0129f80 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0cdc014810 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.665+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0132fa0 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.666+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f0cdc0149c0 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.666+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0062a30 con 0x7f0ce009bea0 2024-02-20T02:14:55.672 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.666+0000 7f0cd67fc640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 0x7f0cc403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.673 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.666+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f0cdc019040 con 0x7f0ce009bea0 2024-02-20T02:14:55.673 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.666+0000 7f0ce5008640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 0x7f0cc403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.673 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.667+0000 7f0ce5008640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 0x7f0cc403a3f0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.673 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.671+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f0cdc00f990 con 0x7f0ce009bea0 2024-02-20T02:14:55.840 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.841+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ce0129e00 con 0x7f0cc4037ee0 2024-02-20T02:14:55.841 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.842+0000 7f0cd67fc640 1 -- 172.21.15.125:0/631272949 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f0cd0001250 con 0x7f0cc4037ee0 2024-02-20T02:14:55.841 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 1 to deep-scrub 2024-02-20T02:14:55.844 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 msgr2=0x7f0cc403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 0x7f0cc403a3f0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 msgr2=0x7f0ce0135ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 shutdown_connections 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0cc4037ee0 0x7f0cc403a3f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.845 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 --2- 172.21.15.125:0/631272949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ce009bea0 0x7f0ce0135ef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.846 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 >> 172.21.15.125:0/631272949 conn(0x7f0ce009fae0 msgr2=0x7f0ce012a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.846 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 shutdown_connections 2024-02-20T02:14:55.846 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.845+0000 7f0ce7a6d640 1 -- 172.21.15.125:0/631272949 wait complete. 2024-02-20T02:14:55.859 INFO:tasks.ceph:Scrubbing osd.2 2024-02-20T02:14:55.859 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-02-20T02:14:55.971 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.965+0000 7f4540a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.972 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.973+0000 7f4540a2d640 1 Processor -- start 2024-02-20T02:14:55.972 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.973+0000 7f4540a2d640 1 -- start start 2024-02-20T02:14:55.974 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f4540a2d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f453c130c60 0x7f453c131070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f4540a2d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c1307a0 con 0x7f453c06da80 2024-02-20T02:14:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f4540a2d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c09bc20 con 0x7f453c130c60 2024-02-20T02:14:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453a59c640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f453c130c60 0x7f453c131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f4539d9b640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f453c06da80 0x7f453c06de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45630/0 (socket says 172.21.15.125:45630) 2024-02-20T02:14:55.975 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f4539d9b640 1 -- 172.21.15.125:0/3406243141 learned_addr learned my addr 172.21.15.125:0/3406243141 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453959a640 1 -- 172.21.15.125:0/3406243141 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126072741 0 0) 0x7f453c1307a0 con 0x7f453c06da80 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453959a640 1 -- 172.21.15.125:0/3406243141 --> v1:172.21.15.125:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45240034f0 con 0x7f453c06da80 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453a59c640 1 -- 172.21.15.125:0/3406243141 >> v1:172.21.15.125:6789/0 conn(0x7f453c06da80 legacy=0x7f453c06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453a59c640 1 -- 172.21.15.125:0/3406243141 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c09be40 con 0x7f453c130c60 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.974+0000 7f453a59c640 1 --2- 172.21.15.125:0/3406243141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c130c60 0x7f453c131070 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b892be88729bd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f453959a640 1 -- 172.21.15.125:0/3406243141 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f452c009ce0 con 0x7f453c130c60 2024-02-20T02:14:55.976 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f453959a640 1 -- 172.21.15.125:0/3406243141 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f452c009e40 con 0x7f453c130c60 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f453959a640 1 -- 172.21.15.125:0/3406243141 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f452c00e360 con 0x7f453c130c60 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 -- 172.21.15.125:0/3406243141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c130c60 msgr2=0x7f453c131070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 --2- 172.21.15.125:0/3406243141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c130c60 0x7f453c131070 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 -- 172.21.15.125:0/3406243141 shutdown_connections 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 --2- 172.21.15.125:0/3406243141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c130c60 0x7f453c131070 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 -- 172.21.15.125:0/3406243141 >> 172.21.15.125:0/3406243141 conn(0x7f453c09fae0 msgr2=0x7f453c09fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 -- 172.21.15.125:0/3406243141 shutdown_connections 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.975+0000 7f4540a2d640 1 -- 172.21.15.125:0/3406243141 wait complete. 2024-02-20T02:14:55.977 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.976+0000 7f4540a2d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f4540a2d640 1 Processor -- start 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f4540a2d640 1 -- start start 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f4540a2d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f4540a2d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c123b30 con 0x7f453c06da80 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f453a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f453a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39476/0 (socket says 172.21.15.125:39476) 2024-02-20T02:14:55.981 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.977+0000 7f453a59c640 1 -- 172.21.15.125:0/1401321862 learned_addr learned my addr 172.21.15.125:0/1401321862 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f453a59c640 1 -- 172.21.15.125:0/1401321862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c09ae20 con 0x7f453c06da80 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f453a59c640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 crc :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f452c014410 con 0x7f453c06da80 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f452c014570 con 0x7f453c06da80 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c123820 con 0x7f453c06da80 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f452c014810 con 0x7f453c06da80 2024-02-20T02:14:55.982 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.978+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c09a560 con 0x7f453c06da80 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f453c1246d0 con 0x7f453c06da80 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f452c0149c0 con 0x7f453c06da80 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 0x7f451c03a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f452c048c40 con 0x7f453c06da80 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 --2- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 0x7f451c0400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 --> v2:172.21.15.204:6811/774159808 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f451c040740 con 0x7f451c03dc20 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=245) v2 ==== 24+0+0 (crc 0 0 0) 0x7f452c04b0b0 con 0x7f453c06da80 2024-02-20T02:14:55.983 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.979+0000 7f4539d9b640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 0x7f451c03a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:55.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.980+0000 7f453ad9d640 1 --2- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 0x7f451c0400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:14:55.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.980+0000 7f4539d9b640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 0x7f451c03a3f0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.984 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.982+0000 7f453ad9d640 1 --2- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 0x7f451c0400b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:55.985 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.983+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== osd.2 v2:172.21.15.204:6811/774159808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f451c040740 con 0x7f451c03dc20 2024-02-20T02:14:55.985 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.986+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 --> v2:172.21.15.204:6811/774159808 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f453c130eb0 con 0x7f451c03dc20 2024-02-20T02:14:55.986 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f452affd640 1 -- 172.21.15.125:0/1401321862 <== osd.2 v2:172.21.15.204:6811/774159808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7f453c130eb0 con 0x7f451c03dc20 2024-02-20T02:14:55.987 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-20T02:14:55.987 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "" 2024-02-20T02:14:55.987 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-20T02:14:55.989 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 msgr2=0x7f451c0400b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.989 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 0x7f451c0400b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.989 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 msgr2=0x7f451c03a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.990 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 0x7f451c03a3f0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.990 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 msgr2=0x7f453c11d1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:55.990 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 crc :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.990 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 shutdown_connections 2024-02-20T02:14:55.991 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> v2:172.21.15.204:6811/774159808 conn(0x7f451c03dc20 0x7f451c0400b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.991 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f451c037ee0 0x7f451c03a3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.991 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 --2- 172.21.15.125:0/1401321862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f453c06da80 0x7f453c11d1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:55.991 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 >> 172.21.15.125:0/1401321862 conn(0x7f453c09fae0 msgr2=0x7f453c12f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:55.992 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 shutdown_connections 2024-02-20T02:14:55.992 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:55.988+0000 7f4540a2d640 1 -- 172.21.15.125:0/1401321862 wait complete. 2024-02-20T02:14:55.999 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-02-20T02:14:56.107 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.104+0000 7f059f624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.108 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.109+0000 7f059f624640 1 Processor -- start 2024-02-20T02:14:56.108 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059f624640 1 -- start start 2024-02-20T02:14:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059f624640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f059809bc40 0x7f0598099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059f624640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0598099a70 con 0x7f059809a3e0 2024-02-20T02:14:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059f624640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059809ae60 con 0x7f059809bc40 2024-02-20T02:14:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059d3c0640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f059809bc40 0x7f0598099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.110 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059cbbf640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f059809a3e0 0x7f0598131890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45636/0 (socket says 172.21.15.125:45636) 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.110+0000 7f059cbbf640 1 -- 172.21.15.125:0/1574223202 learned_addr learned my addr 172.21.15.125:0/1574223202 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f059d3c0640 1 -- 172.21.15.125:0/1574223202 >> v1:172.21.15.125:6789/0 conn(0x7f059809a3e0 legacy=0x7f0598131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f059d3c0640 1 -- 172.21.15.125:0/1574223202 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059809b4d0 con 0x7f059809bc40 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f059d3c0640 1 --2- 172.21.15.125:0/1574223202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809bc40 0x7f0598099ea0 crc :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594cda772403af3e server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f0597fff640 1 -- 172.21.15.125:0/1574223202 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f058800e040 con 0x7f059809bc40 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f0597fff640 1 -- 172.21.15.125:0/1574223202 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0588007670 con 0x7f059809bc40 2024-02-20T02:14:56.111 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.111+0000 7f0597fff640 1 -- 172.21.15.125:0/1574223202 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0588007910 con 0x7f059809bc40 2024-02-20T02:14:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 -- 172.21.15.125:0/1574223202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809bc40 msgr2=0x7f0598099ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.114 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 --2- 172.21.15.125:0/1574223202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809bc40 0x7f0598099ea0 crc :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 -- 172.21.15.125:0/1574223202 shutdown_connections 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 --2- 172.21.15.125:0/1574223202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809bc40 0x7f0598099ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 -- 172.21.15.125:0/1574223202 >> 172.21.15.125:0/1574223202 conn(0x7f059809fae0 msgr2=0x7f059809fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 -- 172.21.15.125:0/1574223202 shutdown_connections 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 1 -- 172.21.15.125:0/1574223202 wait complete. 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.112+0000 7f059f624640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.113+0000 7f059f624640 1 Processor -- start 2024-02-20T02:14:56.115 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.113+0000 7f059f624640 1 -- start start 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.113+0000 7f059f624640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.113+0000 7f059f624640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059811f060 con 0x7f059809a3e0 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.113+0000 7f059d3c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059d3c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39482/0 (socket says 172.21.15.125:39482) 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059d3c0640 1 -- 172.21.15.125:0/1999817714 learned_addr learned my addr 172.21.15.125:0/1999817714 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059d3c0640 1 -- 172.21.15.125:0/1999817714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059806bfc0 con 0x7f059809a3e0 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059d3c0640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 crc :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.116 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f058800e040 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0588012a70 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059811cb00 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0588012d10 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.114+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059811da60 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.115+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f05880153f0 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.115+0000 7f0595ffb640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 0x7f058403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.115+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f0588048d40 con 0x7f059809a3e0 2024-02-20T02:14:56.117 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.115+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059806bfc0 con 0x7f059809a3e0 2024-02-20T02:14:56.118 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.115+0000 7f059cbbf640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 0x7f058403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.118 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.116+0000 7f059cbbf640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 0x7f058403a3f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.118 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.120+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f059806bfc0 con 0x7f059809a3e0 2024-02-20T02:14:56.301 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.302+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f059809a7f0 con 0x7f0584037ee0 2024-02-20T02:14:56.302 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.303+0000 7f0595ffb640 1 -- 172.21.15.125:0/1999817714 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f058c001250 con 0x7f0584037ee0 2024-02-20T02:14:56.302 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 2 to deep-scrub 2024-02-20T02:14:56.304 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 msgr2=0x7f058403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 0x7f058403a3f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 msgr2=0x7f059811aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 crc :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 shutdown_connections 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0584037ee0 0x7f058403a3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 --2- 172.21.15.125:0/1999817714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059809a3e0 0x7f059811aee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 >> 172.21.15.125:0/1999817714 conn(0x7f059809fae0 msgr2=0x7f05981b9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.305 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 shutdown_connections 2024-02-20T02:14:56.306 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.305+0000 7f059f624640 1 -- 172.21.15.125:0/1999817714 wait complete. 2024-02-20T02:14:56.318 INFO:tasks.ceph:Scrubbing osd.3 2024-02-20T02:14:56.318 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-02-20T02:14:56.428 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.422+0000 7fd09202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.429 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.430+0000 7fd09202d640 1 Processor -- start 2024-02-20T02:14:56.429 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.430+0000 7fd09202d640 1 -- start start 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.430+0000 7fd09202d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fd08c133230 0x7fd08c135650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.430+0000 7fd09202d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c09bf80 con 0x7fd08c130470 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.430+0000 7fd09202d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c12fec0 con 0x7fd08c133230 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08affd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7fd08c133230 0x7fd08c135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08b7fe640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7fd08c130470 0x7fd08c130880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45648/0 (socket says 172.21.15.125:45648) 2024-02-20T02:14:56.431 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08b7fe640 1 -- 172.21.15.125:0/565344106 learned_addr learned my addr 172.21.15.125:0/565344106 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08affd640 1 -- 172.21.15.125:0/565344106 >> v1:172.21.15.125:6789/0 conn(0x7fd08c130470 legacy=0x7fd08c130880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08affd640 1 -- 172.21.15.125:0/565344106 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c130160 con 0x7fd08c133230 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08a7fc640 1 -- 172.21.15.125:0/565344106 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485324690 0 0) 0x7fd08c09bf80 con 0x7fd08c130470 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08affd640 1 --2- 172.21.15.125:0/565344106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c133230 0x7fd08c135650 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8380041e8767ce server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08a7fc640 1 -- 172.21.15.125:0/565344106 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd07c009ce0 con 0x7fd08c133230 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08a7fc640 1 -- 172.21.15.125:0/565344106 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd07c009e40 con 0x7fd08c133230 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.431+0000 7fd08a7fc640 1 -- 172.21.15.125:0/565344106 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd07c00f470 con 0x7fd08c133230 2024-02-20T02:14:56.432 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 -- 172.21.15.125:0/565344106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c133230 msgr2=0x7fd08c135650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 --2- 172.21.15.125:0/565344106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c133230 0x7fd08c135650 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 -- 172.21.15.125:0/565344106 shutdown_connections 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 --2- 172.21.15.125:0/565344106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c133230 0x7fd08c135650 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 -- 172.21.15.125:0/565344106 >> 172.21.15.125:0/565344106 conn(0x7fd08c09fae0 msgr2=0x7fd08c09fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 -- 172.21.15.125:0/565344106 shutdown_connections 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 1 -- 172.21.15.125:0/565344106 wait complete. 2024-02-20T02:14:56.433 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.432+0000 7fd09202d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.433+0000 7fd09202d640 1 Processor -- start 2024-02-20T02:14:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.433+0000 7fd09202d640 1 -- start start 2024-02-20T02:14:56.436 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd09202d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd09202d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c11a650 con 0x7fd08c130470 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd08b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd08b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39500/0 (socket says 172.21.15.125:39500) 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd08b7fe640 1 -- 172.21.15.125:0/2214298704 learned_addr learned my addr 172.21.15.125:0/2214298704 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd08b7fe640 1 -- 172.21.15.125:0/2214298704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c129f60 con 0x7fd08c130470 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd08b7fe640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.437 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd080004af0 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd080004c50 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.434+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c1183c0 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.435+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd080007120 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.435+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c119770 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.435+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd08c12a0f0 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.435+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7fd0800072d0 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd073fff640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 0x7fd07803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7fd080047700 con 0x7fd08c130470 2024-02-20T02:14:56.438 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd073fff640 1 --2- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 0x7fd0780400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 --> v2:172.21.15.204:6804/409650538 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd078040740 con 0x7fd07803dc20 2024-02-20T02:14:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_get_version_reply(handle=1 version=245) v2 ==== 24+0+0 (crc 0 0 0) 0x7fd080049b70 con 0x7fd08c130470 2024-02-20T02:14:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd08affd640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 0x7fd07803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd08bfff640 1 --2- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 0x7fd0780400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T02:14:56.439 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.436+0000 7fd08affd640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 0x7fd07803a3f0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.438+0000 7fd08bfff640 1 --2- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 0x7fd0780400b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.440 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.439+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== osd.3 v2:172.21.15.204:6804/409650538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fd078040740 con 0x7fd07803dc20 2024-02-20T02:14:56.441 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.443+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 --> v2:172.21.15.204:6804/409650538 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd08c09ba30 con 0x7fd07803dc20 2024-02-20T02:14:56.443 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd073fff640 1 -- 172.21.15.125:0/2214298704 <== osd.3 v2:172.21.15.204:6804/409650538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7fd08400a820 con 0x7fd07803dc20 2024-02-20T02:14:56.443 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-20T02:14:56.443 INFO:teuthology.orchestra.run.smithi125.stdout: "success": "" 2024-02-20T02:14:56.443 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-20T02:14:56.444 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 msgr2=0x7fd0780400b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.444 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 0x7fd0780400b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 msgr2=0x7fd07803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 0x7fd07803a3f0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 msgr2=0x7fd08c0710b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 shutdown_connections 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> v2:172.21.15.204:6804/409650538 conn(0x7fd07803dc20 0x7fd0780400b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7fd078037ee0 0x7fd07803a3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 --2- 172.21.15.125:0/2214298704 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd08c130470 0x7fd08c0710b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.445 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 >> 172.21.15.125:0/2214298704 conn(0x7fd08c09fae0 msgr2=0x7fd08c09a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.446 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 shutdown_connections 2024-02-20T02:14:56.446 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.444+0000 7fd09202d640 1 -- 172.21.15.125:0/2214298704 wait complete. 2024-02-20T02:14:56.453 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-02-20T02:14:56.560 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.557+0000 7f5e9b11d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.562+0000 7f5e9b11d640 1 Processor -- start 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.562+0000 7f5e9b11d640 1 -- start start 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.562+0000 7f5e9b11d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5e9409e170 0x7f5e940a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.562+0000 7f5e9b11d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e940a0260 con 0x7f5e940a0c60 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.562+0000 7f5e9b11d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9409d450 con 0x7f5e9409e170 2024-02-20T02:14:56.564 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e98eb9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f5e9409e170 0x7f5e940a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e93fff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f5e940a0c60 0x7f5e94138510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45658/0 (socket says 172.21.15.125:45658) 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e93fff640 1 -- 172.21.15.125:0/3509218677 learned_addr learned my addr 172.21.15.125:0/3509218677 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e98eb9640 1 -- 172.21.15.125:0/3509218677 >> v1:172.21.15.125:6789/0 conn(0x7f5e940a0c60 legacy=0x7f5e94138510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e98eb9640 1 -- 172.21.15.125:0/3509218677 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9409da90 con 0x7f5e9409e170 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e98eb9640 1 --2- 172.21.15.125:0/3509218677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e940a0720 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402f72e75584b435 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e937fe640 1 -- 172.21.15.125:0/3509218677 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e8c00e040 con 0x7f5e9409e170 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e937fe640 1 -- 172.21.15.125:0/3509218677 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e8c007670 con 0x7f5e9409e170 2024-02-20T02:14:56.565 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.563+0000 7f5e937fe640 1 -- 172.21.15.125:0/3509218677 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e8c007910 con 0x7f5e9409e170 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3509218677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 msgr2=0x7f5e940a0720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3509218677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e940a0720 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3509218677 shutdown_connections 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3509218677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e940a0720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3509218677 >> 172.21.15.125:0/3509218677 conn(0x7f5e940a29c0 msgr2=0x7f5e940a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3509218677 shutdown_connections 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3509218677 wait complete. 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.564+0000 7f5e9b11d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.565+0000 7f5e9b11d640 1 Processor -- start 2024-02-20T02:14:56.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.565+0000 7f5e9b11d640 1 -- start start 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e9b11d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e9b11d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e941c5310 con 0x7f5e9409e170 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e98eb9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e98eb9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39516/0 (socket says 172.21.15.125:39516) 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e98eb9640 1 -- 172.21.15.125:0/3213068435 learned_addr learned my addr 172.21.15.125:0/3213068435 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e98eb9640 1 -- 172.21.15.125:0/3213068435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e941ab000 con 0x7f5e9409e170 2024-02-20T02:14:56.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e98eb9640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e8c00e040 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e8c012a90 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e8c012d30 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.566+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e941c5540 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.567+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e941ab660 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.567+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f5e8c0153f0 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.567+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e94062a30 con 0x7f5e9409e170 2024-02-20T02:14:56.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.568+0000 7f5e917fa640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 0x7f5e7403a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.568+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f5e8c049020 con 0x7f5e9409e170 2024-02-20T02:14:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.568+0000 7f5e93fff640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 0x7f5e7403a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.568+0000 7f5e93fff640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 0x7f5e7403a3f0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.571 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.572+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f5e8c00fc80 con 0x7f5e9409e170 2024-02-20T02:14:56.743 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.744+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e941acd20 con 0x7f5e74037ee0 2024-02-20T02:14:56.744 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.746+0000 7f5e917fa640 1 -- 172.21.15.125:0/3213068435 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7f5e941acd20 con 0x7f5e74037ee0 2024-02-20T02:14:56.744 INFO:teuthology.orchestra.run.smithi125.stderr:instructed osd(s) 3 to deep-scrub 2024-02-20T02:14:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 msgr2=0x7f5e7403a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 0x7f5e7403a3f0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.746 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 msgr2=0x7f5e941c5850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 shutdown_connections 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f5e74037ee0 0x7f5e7403a3f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 --2- 172.21.15.125:0/3213068435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e9409e170 0x7f5e941c5850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 >> 172.21.15.125:0/3213068435 conn(0x7f5e940a29c0 msgr2=0x7f5e9409af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 shutdown_connections 2024-02-20T02:14:56.747 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.747+0000 7f5e9b11d640 1 -- 172.21.15.125:0/3213068435 wait complete. 2024-02-20T02:14:56.758 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:14:56.863 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.861+0000 7f292bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.865+0000 7f292bf5d640 1 Processor -- start 2024-02-20T02:14:56.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.865+0000 7f292bf5d640 1 -- start start 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f292bf5d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f2924130c20 0x7f2924131030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f292bf5d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2924130840 con 0x7f292406da80 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f292bf5d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292406f2f0 con 0x7f2924130c20 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f2929cf9640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f2924130c20 0x7f2924131030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f29294f8640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f292406da80 0x7f292406de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:45666/0 (socket says 172.21.15.125:45666) 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f29294f8640 1 -- 172.21.15.125:0/2258397416 learned_addr learned my addr 172.21.15.125:0/2258397416 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:14:56.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f2929cf9640 1 -- 172.21.15.125:0/2258397416 >> v1:172.21.15.125:6789/0 conn(0x7f292406da80 legacy=0x7f292406de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.866+0000 7f2929cf9640 1 -- 172.21.15.125:0/2258397416 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924067a90 con 0x7f2924130c20 2024-02-20T02:14:56.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f2929cf9640 1 --2- 172.21.15.125:0/2258397416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2924130c20 0x7f2924131030 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4410f07757f3b3d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f2928cf7640 1 -- 172.21.15.125:0/2258397416 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2920009ce0 con 0x7f2924130c20 2024-02-20T02:14:56.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f2928cf7640 1 -- 172.21.15.125:0/2258397416 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2920009e40 con 0x7f2924130c20 2024-02-20T02:14:56.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f2928cf7640 1 -- 172.21.15.125:0/2258397416 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f292000e360 con 0x7f2924130c20 2024-02-20T02:14:56.869 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f292bf5d640 1 -- 172.21.15.125:0/2258397416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2924130c20 msgr2=0x7f2924131030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:56.869 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f292bf5d640 1 --2- 172.21.15.125:0/2258397416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2924130c20 0x7f2924131030 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f292bf5d640 1 -- 172.21.15.125:0/2258397416 shutdown_connections 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f292bf5d640 1 --2- 172.21.15.125:0/2258397416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2924130c20 0x7f2924131030 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.867+0000 7f292bf5d640 1 -- 172.21.15.125:0/2258397416 >> 172.21.15.125:0/2258397416 conn(0x7f292409fae0 msgr2=0x7f292409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.868+0000 7f292bf5d640 1 -- 172.21.15.125:0/2258397416 shutdown_connections 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.868+0000 7f292bf5d640 1 -- 172.21.15.125:0/2258397416 wait complete. 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.868+0000 7f292bf5d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f292bf5d640 1 Processor -- start 2024-02-20T02:14:56.870 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f292bf5d640 1 -- start start 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f292bf5d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f292bf5d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292411da80 con 0x7f292406da80 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f2929cf9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f2929cf9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:39534/0 (socket says 172.21.15.125:39534) 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.869+0000 7f2929cf9640 1 -- 172.21.15.125:0/3543132938 learned_addr learned my addr 172.21.15.125:0/3543132938 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f2929cf9640 1 -- 172.21.15.125:0/3543132938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292411dd40 con 0x7f292406da80 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f2929cf9640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2920014540 con 0x7f292406da80 2024-02-20T02:14:56.871 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f29200146a0 con 0x7f292406da80 2024-02-20T02:14:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292409ae90 con 0x7f292406da80 2024-02-20T02:14:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2920014940 con 0x7f292406da80 2024-02-20T02:14:56.872 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.870+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292411e5d0 con 0x7f292406da80 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.871+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f2920014af0 con 0x7f292406da80 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.871+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2924062a30 con 0x7f292406da80 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.871+0000 7f291a7fc640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 0x7f290803a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.871+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f2920019040 con 0x7f292406da80 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.872+0000 7f29294f8640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 0x7f290803a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:14:56.873 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.872+0000 7f29294f8640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 0x7f290803a420 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:14:56.874 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:56.876+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f2920018190 con 0x7f292406da80 2024-02-20T02:14:57.076 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.077+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29240a17b0 con 0x7f2908037f10 2024-02-20T02:14:57.079 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.080+0000 7f291a7fc640 1 -- 172.21.15.125:0/3543132938 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107055 (crc 0 0 0) 0x7f29140011a0 con 0x7f2908037f10 2024-02-20T02:14:57.080 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:14:57.080 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:14:57.082 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 msgr2=0x7f290803a420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:57.082 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 0x7f290803a420 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:57.082 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 msgr2=0x7f29241c6ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 shutdown_connections 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f2908037f10 0x7f290803a420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 --2- 172.21.15.125:0/3543132938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f292406da80 0x7f29241c6ac0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 >> 172.21.15.125:0/3543132938 conn(0x7f292409fae0 msgr2=0x7f292409ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 shutdown_connections 2024-02-20T02:14:57.083 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:14:57.082+0000 7f292bf5d640 1 -- 172.21.15.125:0/3543132938 wait complete. 2024-02-20T02:14:57.092 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-02-20T02:14:57.011946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":151416,"kb_used_data":10328,"kb_used_omap":0,"kb_used_meta":141056,"kb_avail":419278984,"statfs":{"total":429496729600,"available":429341679616,"internally_reserved":0,"allocated":10575872,"data_stored":4641282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":144441344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516453"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.230762+0000","last_change":"2024-02-20T02:14:44.190037+0000","last_active":"2024-02-20T02:14:52.230762+0000","last_peered":"2024-02-20T02:14:52.230762+0000","last_clean":"2024-02-20T02:14:52.230762+0000","last_became_active":"2024-02-20T02:14:44.170164+0000","last_became_peered":"2024-02-20T02:14:44.170164+0000","last_unstale":"2024-02-20T02:14:52.230762+0000","last_undegraded":"2024-02-20T02:14:52.230762+0000","last_fullsized":"2024-02-20T02:14:52.230762+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.284457+0000","last_change":"2024-02-20T02:14:44.201273+0000","last_active":"2024-02-20T02:14:52.284457+0000","last_peered":"2024-02-20T02:14:52.284457+0000","last_clean":"2024-02-20T02:14:52.284457+0000","last_became_active":"2024-02-20T02:14:44.189022+0000","last_became_peered":"2024-02-20T02:14:44.189022+0000","last_unstale":"2024-02-20T02:14:52.284457+0000","last_undegraded":"2024-02-20T02:14:52.284457+0000","last_fullsized":"2024-02-20T02:14:52.284457+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.239884+0000","last_change":"2024-02-20T02:14:44.181287+0000","last_active":"2024-02-20T02:14:52.239884+0000","last_peered":"2024-02-20T02:14:52.239884+0000","last_clean":"2024-02-20T02:14:52.239884+0000","last_became_active":"2024-02-20T02:14:44.173755+0000","last_became_peered":"2024-02-20T02:14:44.173755+0000","last_unstale":"2024-02-20T02:14:52.239884+0000","last_undegraded":"2024-02-20T02:14:52.239884+0000","last_fullsized":"2024-02-20T02:14:52.239884+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.129357+0000","last_change":"2024-02-20T02:14:44.211892+0000","last_active":"2024-02-20T02:14:52.129357+0000","last_peered":"2024-02-20T02:14:52.129357+0000","last_clean":"2024-02-20T02:14:52.129357+0000","last_became_active":"2024-02-20T02:14:44.203048+0000","last_became_peered":"2024-02-20T02:14:44.203048+0000","last_unstale":"2024-02-20T02:14:52.129357+0000","last_undegraded":"2024-02-20T02:14:52.129357+0000","last_fullsized":"2024-02-20T02:14:52.129357+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.336510+0000","last_change":"2024-02-20T02:14:44.206069+0000","last_active":"2024-02-20T02:14:52.336510+0000","last_peered":"2024-02-20T02:14:52.336510+0000","last_clean":"2024-02-20T02:14:52.336510+0000","last_became_active":"2024-02-20T02:14:44.201917+0000","last_became_peered":"2024-02-20T02:14:44.201917+0000","last_unstale":"2024-02-20T02:14:52.336510+0000","last_undegraded":"2024-02-20T02:14:52.336510+0000","last_fullsized":"2024-02-20T02:14:52.336510+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.329429+0000","last_change":"2024-02-20T02:14:44.187033+0000","last_active":"2024-02-20T02:14:52.329429+0000","last_peered":"2024-02-20T02:14:52.329429+0000","last_clean":"2024-02-20T02:14:52.329429+0000","last_became_active":"2024-02-20T02:14:44.176812+0000","last_became_peered":"2024-02-20T02:14:44.176812+0000","last_unstale":"2024-02-20T02:14:52.329429+0000","last_undegraded":"2024-02-20T02:14:52.329429+0000","last_fullsized":"2024-02-20T02:14:52.329429+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.331945+0000","last_change":"2024-02-20T02:14:44.180791+0000","last_active":"2024-02-20T02:14:52.331945+0000","last_peered":"2024-02-20T02:14:52.331945+0000","last_clean":"2024-02-20T02:14:52.331945+0000","last_became_active":"2024-02-20T02:14:44.173660+0000","last_became_peered":"2024-02-20T02:14:44.173660+0000","last_unstale":"2024-02-20T02:14:52.331945+0000","last_undegraded":"2024-02-20T02:14:52.331945+0000","last_fullsized":"2024-02-20T02:14:52.331945+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.347304+0000","last_change":"2024-02-20T02:14:44.204673+0000","last_active":"2024-02-20T02:14:52.347304+0000","last_peered":"2024-02-20T02:14:52.347304+0000","last_clean":"2024-02-20T02:14:52.347304+0000","last_became_active":"2024-02-20T02:14:44.193652+0000","last_became_peered":"2024-02-20T02:14:44.193652+0000","last_unstale":"2024-02-20T02:14:52.347304+0000","last_undegraded":"2024-02-20T02:14:52.347304+0000","last_fullsized":"2024-02-20T02:14:52.347304+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.137633+0000","last_change":"2024-02-20T02:14:44.208695+0000","last_active":"2024-02-20T02:14:52.137633+0000","last_peered":"2024-02-20T02:14:52.137633+0000","last_clean":"2024-02-20T02:14:52.137633+0000","last_became_active":"2024-02-20T02:14:44.199375+0000","last_became_peered":"2024-02-20T02:14:44.199375+0000","last_unstale":"2024-02-20T02:14:52.137633+0000","last_undegraded":"2024-02-20T02:14:52.137633+0000","last_fullsized":"2024-02-20T02:14:52.137633+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.243478+0000","last_change":"2024-02-20T02:14:44.203898+0000","last_active":"2024-02-20T02:14:52.243478+0000","last_peered":"2024-02-20T02:14:52.243478+0000","last_clean":"2024-02-20T02:14:52.243478+0000","last_became_active":"2024-02-20T02:14:44.200473+0000","last_became_peered":"2024-02-20T02:14:44.200473+0000","last_unstale":"2024-02-20T02:14:52.243478+0000","last_undegraded":"2024-02-20T02:14:52.243478+0000","last_fullsized":"2024-02-20T02:14:52.243478+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.132010+0000","last_change":"2024-02-20T02:14:44.196513+0000","last_active":"2024-02-20T02:14:52.132010+0000","last_peered":"2024-02-20T02:14:52.132010+0000","last_clean":"2024-02-20T02:14:52.132010+0000","last_became_active":"2024-02-20T02:14:44.190055+0000","last_became_peered":"2024-02-20T02:14:44.190055+0000","last_unstale":"2024-02-20T02:14:52.132010+0000","last_undegraded":"2024-02-20T02:14:52.132010+0000","last_fullsized":"2024-02-20T02:14:52.132010+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.311070+0000","last_change":"2024-02-20T02:14:44.195328+0000","last_active":"2024-02-20T02:14:52.311070+0000","last_peered":"2024-02-20T02:14:52.311070+0000","last_clean":"2024-02-20T02:14:52.311070+0000","last_became_active":"2024-02-20T02:14:44.184534+0000","last_became_peered":"2024-02-20T02:14:44.184534+0000","last_unstale":"2024-02-20T02:14:52.311070+0000","last_undegraded":"2024-02-20T02:14:52.311070+0000","last_fullsized":"2024-02-20T02:14:52.311070+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.100359+0000","last_change":"2024-02-20T02:14:44.201331+0000","last_active":"2024-02-20T02:14:52.100359+0000","last_peered":"2024-02-20T02:14:52.100359+0000","last_clean":"2024-02-20T02:14:52.100359+0000","last_became_active":"2024-02-20T02:14:44.192136+0000","last_became_peered":"2024-02-20T02:14:44.192136+0000","last_unstale":"2024-02-20T02:14:52.100359+0000","last_undegraded":"2024-02-20T02:14:52.100359+0000","last_fullsized":"2024-02-20T02:14:52.100359+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.139102+0000","last_change":"2024-02-20T02:14:44.258170+0000","last_active":"2024-02-20T02:14:52.139102+0000","last_peered":"2024-02-20T02:14:52.139102+0000","last_clean":"2024-02-20T02:14:52.139102+0000","last_became_active":"2024-02-20T02:14:44.254178+0000","last_became_peered":"2024-02-20T02:14:44.254178+0000","last_unstale":"2024-02-20T02:14:52.139102+0000","last_undegraded":"2024-02-20T02:14:52.139102+0000","last_fullsized":"2024-02-20T02:14:52.139102+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.257065+0000","last_change":"2024-02-20T02:14:44.253515+0000","last_active":"2024-02-20T02:14:52.257065+0000","last_peered":"2024-02-20T02:14:52.257065+0000","last_clean":"2024-02-20T02:14:52.257065+0000","last_became_active":"2024-02-20T02:14:44.250168+0000","last_became_peered":"2024-02-20T02:14:44.250168+0000","last_unstale":"2024-02-20T02:14:52.257065+0000","last_undegraded":"2024-02-20T02:14:52.257065+0000","last_fullsized":"2024-02-20T02:14:52.257065+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.308718+0000","last_change":"2024-02-20T02:14:44.212643+0000","last_active":"2024-02-20T02:14:52.308718+0000","last_peered":"2024-02-20T02:14:52.308718+0000","last_clean":"2024-02-20T02:14:52.308718+0000","last_became_active":"2024-02-20T02:14:44.208652+0000","last_became_peered":"2024-02-20T02:14:44.208652+0000","last_unstale":"2024-02-20T02:14:52.308718+0000","last_undegraded":"2024-02-20T02:14:52.308718+0000","last_fullsized":"2024-02-20T02:14:52.308718+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.300839+0000","last_change":"2024-02-20T02:14:44.206611+0000","last_active":"2024-02-20T02:14:52.300839+0000","last_peered":"2024-02-20T02:14:52.300839+0000","last_clean":"2024-02-20T02:14:52.300839+0000","last_became_active":"2024-02-20T02:14:44.197136+0000","last_became_peered":"2024-02-20T02:14:44.197136+0000","last_unstale":"2024-02-20T02:14:52.300839+0000","last_undegraded":"2024-02-20T02:14:52.300839+0000","last_fullsized":"2024-02-20T02:14:52.300839+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.125869+0000","last_change":"2024-02-20T02:14:44.224136+0000","last_active":"2024-02-20T02:14:52.125869+0000","last_peered":"2024-02-20T02:14:52.125869+0000","last_clean":"2024-02-20T02:14:52.125869+0000","last_became_active":"2024-02-20T02:14:44.218175+0000","last_became_peered":"2024-02-20T02:14:44.218175+0000","last_unstale":"2024-02-20T02:14:52.125869+0000","last_undegraded":"2024-02-20T02:14:52.125869+0000","last_fullsized":"2024-02-20T02:14:52.125869+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.222390+0000","last_change":"2024-02-20T02:14:44.182566+0000","last_active":"2024-02-20T02:14:52.222390+0000","last_peered":"2024-02-20T02:14:52.222390+0000","last_clean":"2024-02-20T02:14:52.222390+0000","last_became_active":"2024-02-20T02:14:44.175757+0000","last_became_peered":"2024-02-20T02:14:44.175757+0000","last_unstale":"2024-02-20T02:14:52.222390+0000","last_undegraded":"2024-02-20T02:14:52.222390+0000","last_fullsized":"2024-02-20T02:14:52.222390+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.291904+0000","last_change":"2024-02-20T02:14:44.179208+0000","last_active":"2024-02-20T02:14:52.291904+0000","last_peered":"2024-02-20T02:14:52.291904+0000","last_clean":"2024-02-20T02:14:52.291904+0000","last_became_active":"2024-02-20T02:14:44.167479+0000","last_became_peered":"2024-02-20T02:14:44.167479+0000","last_unstale":"2024-02-20T02:14:52.291904+0000","last_undegraded":"2024-02-20T02:14:52.291904+0000","last_fullsized":"2024-02-20T02:14:52.291904+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.230072+0000","last_change":"2024-02-20T02:14:44.192711+0000","last_active":"2024-02-20T02:14:52.230072+0000","last_peered":"2024-02-20T02:14:52.230072+0000","last_clean":"2024-02-20T02:14:52.230072+0000","last_became_active":"2024-02-20T02:14:44.180769+0000","last_became_peered":"2024-02-20T02:14:44.180769+0000","last_unstale":"2024-02-20T02:14:52.230072+0000","last_undegraded":"2024-02-20T02:14:52.230072+0000","last_fullsized":"2024-02-20T02:14:52.230072+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.289179+0000","last_change":"2024-02-20T02:14:44.191580+0000","last_active":"2024-02-20T02:14:52.289179+0000","last_peered":"2024-02-20T02:14:52.289179+0000","last_clean":"2024-02-20T02:14:52.289179+0000","last_became_active":"2024-02-20T02:14:44.182801+0000","last_became_peered":"2024-02-20T02:14:44.182801+0000","last_unstale":"2024-02-20T02:14:52.289179+0000","last_undegraded":"2024-02-20T02:14:52.289179+0000","last_fullsized":"2024-02-20T02:14:52.289179+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.240600+0000","last_change":"2024-02-20T02:14:44.190778+0000","last_active":"2024-02-20T02:14:52.240600+0000","last_peered":"2024-02-20T02:14:52.240600+0000","last_clean":"2024-02-20T02:14:52.240600+0000","last_became_active":"2024-02-20T02:14:44.186481+0000","last_became_peered":"2024-02-20T02:14:44.186481+0000","last_unstale":"2024-02-20T02:14:52.240600+0000","last_undegraded":"2024-02-20T02:14:52.240600+0000","last_fullsized":"2024-02-20T02:14:52.240600+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.122535+0000","last_change":"2024-02-20T02:14:44.246778+0000","last_active":"2024-02-20T02:14:52.122535+0000","last_peered":"2024-02-20T02:14:52.122535+0000","last_clean":"2024-02-20T02:14:52.122535+0000","last_became_active":"2024-02-20T02:14:44.243754+0000","last_became_peered":"2024-02-20T02:14:44.243754+0000","last_unstale":"2024-02-20T02:14:52.122535+0000","last_undegraded":"2024-02-20T02:14:52.122535+0000","last_fullsized":"2024-02-20T02:14:52.122535+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.120182+0000","last_change":"2024-02-20T02:14:44.205105+0000","last_active":"2024-02-20T02:14:52.120182+0000","last_peered":"2024-02-20T02:14:52.120182+0000","last_clean":"2024-02-20T02:14:52.120182+0000","last_became_active":"2024-02-20T02:14:44.194528+0000","last_became_peered":"2024-02-20T02:14:44.194528+0000","last_unstale":"2024-02-20T02:14:52.120182+0000","last_undegraded":"2024-02-20T02:14:52.120182+0000","last_fullsized":"2024-02-20T02:14:52.120182+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.346238+0000","last_change":"2024-02-20T02:14:44.248214+0000","last_active":"2024-02-20T02:14:52.346238+0000","last_peered":"2024-02-20T02:14:52.346238+0000","last_clean":"2024-02-20T02:14:52.346238+0000","last_became_active":"2024-02-20T02:14:44.244704+0000","last_became_peered":"2024-02-20T02:14:44.244704+0000","last_unstale":"2024-02-20T02:14:52.346238+0000","last_undegraded":"2024-02-20T02:14:52.346238+0000","last_fullsized":"2024-02-20T02:14:52.346238+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.082880+0000","last_change":"2024-02-20T02:14:44.238032+0000","last_active":"2024-02-20T02:14:52.082880+0000","last_peered":"2024-02-20T02:14:52.082880+0000","last_clean":"2024-02-20T02:14:52.082880+0000","last_became_active":"2024-02-20T02:14:44.234764+0000","last_became_peered":"2024-02-20T02:14:44.234764+0000","last_unstale":"2024-02-20T02:14:52.082880+0000","last_undegraded":"2024-02-20T02:14:52.082880+0000","last_fullsized":"2024-02-20T02:14:52.082880+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.264898+0000","last_change":"2024-02-20T02:14:44.241835+0000","last_active":"2024-02-20T02:14:52.264898+0000","last_peered":"2024-02-20T02:14:52.264898+0000","last_clean":"2024-02-20T02:14:52.264898+0000","last_became_active":"2024-02-20T02:14:44.238057+0000","last_became_peered":"2024-02-20T02:14:44.238057+0000","last_unstale":"2024-02-20T02:14:52.264898+0000","last_undegraded":"2024-02-20T02:14:52.264898+0000","last_fullsized":"2024-02-20T02:14:52.264898+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.305148+0000","last_change":"2024-02-20T02:14:44.199397+0000","last_active":"2024-02-20T02:14:52.305148+0000","last_peered":"2024-02-20T02:14:52.305148+0000","last_clean":"2024-02-20T02:14:52.305148+0000","last_became_active":"2024-02-20T02:14:44.182848+0000","last_became_peered":"2024-02-20T02:14:44.182848+0000","last_unstale":"2024-02-20T02:14:52.305148+0000","last_undegraded":"2024-02-20T02:14:52.305148+0000","last_fullsized":"2024-02-20T02:14:52.305148+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.313258+0000","last_change":"2024-02-20T02:14:44.202362+0000","last_active":"2024-02-20T02:14:52.313258+0000","last_peered":"2024-02-20T02:14:52.313258+0000","last_clean":"2024-02-20T02:14:52.313258+0000","last_became_active":"2024-02-20T02:14:44.191242+0000","last_became_peered":"2024-02-20T02:14:44.191242+0000","last_unstale":"2024-02-20T02:14:52.313258+0000","last_undegraded":"2024-02-20T02:14:52.313258+0000","last_fullsized":"2024-02-20T02:14:52.313258+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"238'7","reported_seq":42,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.293882+0000","last_change":"2024-02-20T02:14:44.187038+0000","last_active":"2024-02-20T02:14:52.293882+0000","last_peered":"2024-02-20T02:14:52.293882+0000","last_clean":"2024-02-20T02:14:52.293882+0000","last_became_active":"2024-02-20T02:14:44.172077+0000","last_became_peered":"2024-02-20T02:14:44.172077+0000","last_unstale":"2024-02-20T02:14:52.293882+0000","last_undegraded":"2024-02-20T02:14:52.293882+0000","last_fullsized":"2024-02-20T02:14:52.293882+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":29,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.220377+0000","last_change":"2024-02-20T02:14:44.178106+0000","last_active":"2024-02-20T02:14:52.220377+0000","last_peered":"2024-02-20T02:14:52.220377+0000","last_clean":"2024-02-20T02:14:52.220377+0000","last_became_active":"2024-02-20T02:14:44.171906+0000","last_became_peered":"2024-02-20T02:14:44.171906+0000","last_unstale":"2024-02-20T02:14:52.220377+0000","last_undegraded":"2024-02-20T02:14:52.220377+0000","last_fullsized":"2024-02-20T02:14:52.220377+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:43.010974+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":615,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.263831+0000","last_change":"2024-02-20T02:09:44.207250+0000","last_active":"2024-02-20T02:14:52.263831+0000","last_peered":"2024-02-20T02:14:52.263831+0000","last_clean":"2024-02-20T02:14:52.263831+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:14:52.263831+0000","last_undegraded":"2024-02-20T02:14:52.263831+0000","last_fullsized":"2024-02-20T02:14:52.263831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":609,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.227493+0000","last_change":"2024-02-20T02:09:44.213432+0000","last_active":"2024-02-20T02:14:52.227493+0000","last_peered":"2024-02-20T02:14:52.227493+0000","last_clean":"2024-02-20T02:14:52.227493+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:14:52.227493+0000","last_undegraded":"2024-02-20T02:14:52.227493+0000","last_fullsized":"2024-02-20T02:14:52.227493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":621,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.104268+0000","last_change":"2024-02-20T02:09:44.208389+0000","last_active":"2024-02-20T02:14:52.104268+0000","last_peered":"2024-02-20T02:14:52.104268+0000","last_clean":"2024-02-20T02:14:52.104268+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:14:52.104268+0000","last_undegraded":"2024-02-20T02:14:52.104268+0000","last_fullsized":"2024-02-20T02:14:52.104268+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":609,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.213982+0000","last_change":"2024-02-20T02:09:44.196172+0000","last_active":"2024-02-20T02:14:52.213982+0000","last_peered":"2024-02-20T02:14:52.213982+0000","last_clean":"2024-02-20T02:14:52.213982+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:14:52.213982+0000","last_undegraded":"2024-02-20T02:14:52.213982+0000","last_fullsized":"2024-02-20T02:14:52.213982+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":615,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.352924+0000","last_change":"2024-02-20T02:09:44.200962+0000","last_active":"2024-02-20T02:14:52.352924+0000","last_peered":"2024-02-20T02:14:52.352924+0000","last_clean":"2024-02-20T02:14:52.352924+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:14:52.352924+0000","last_undegraded":"2024-02-20T02:14:52.352924+0000","last_fullsized":"2024-02-20T02:14:52.352924+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":615,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.335346+0000","last_change":"2024-02-20T02:09:44.195673+0000","last_active":"2024-02-20T02:14:52.335346+0000","last_peered":"2024-02-20T02:14:52.335346+0000","last_clean":"2024-02-20T02:14:52.335346+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:14:52.335346+0000","last_undegraded":"2024-02-20T02:14:52.335346+0000","last_fullsized":"2024-02-20T02:14:52.335346+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":615,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.317394+0000","last_change":"2024-02-20T02:09:44.204087+0000","last_active":"2024-02-20T02:14:52.317394+0000","last_peered":"2024-02-20T02:14:52.317394+0000","last_clean":"2024-02-20T02:14:52.317394+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:14:52.317394+0000","last_undegraded":"2024-02-20T02:14:52.317394+0000","last_fullsized":"2024-02-20T02:14:52.317394+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":615,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.346211+0000","last_change":"2024-02-20T02:09:44.214794+0000","last_active":"2024-02-20T02:14:52.346211+0000","last_peered":"2024-02-20T02:14:52.346211+0000","last_clean":"2024-02-20T02:14:52.346211+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:14:52.346211+0000","last_undegraded":"2024-02-20T02:14:52.346211+0000","last_fullsized":"2024-02-20T02:14:52.346211+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:43.147664+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":757,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-02-20T02:14:52.082043+0000","last_change":"2024-02-20T02:09:40.192721+0000","last_active":"2024-02-20T02:14:52.082043+0000","last_peered":"2024-02-20T02:14:52.082043+0000","last_clean":"2024-02-20T02:14:52.082043+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:14:52.082043+0000","last_undegraded":"2024-02-20T02:14:52.082043+0000","last_fullsized":"2024-02-20T02:14:52.082043+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:09:39.134569+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":42324,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815276,"statfs":{"total":107374182400,"available":107330842624,"internally_reserved":0,"allocated":2961408,"data_stored":1471086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":37560,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":35264,"kb_avail":104820040,"statfs":{"total":107374182400,"available":107335720960,"internally_reserved":0,"allocated":2342912,"data_stored":859987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36110336},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":41716,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815884,"statfs":{"total":107374182400,"available":107331465216,"internally_reserved":0,"allocated":2338816,"data_stored":859859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705979,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":29816,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104827784,"statfs":{"total":107374182400,"available":107343650816,"internally_reserved":0,"allocated":2932736,"data_stored":1450350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:14:57.096 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.097 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.098 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.099 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.100 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.101 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-02-20T02:14:43.010974+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.102 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.103 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.103 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-02-20T02:09:43.147664+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=43, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.103 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-02-20T02:09:39.134569+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=9, tm_sec=39, tm_wday=1, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=20, tm_hour=2, tm_min=14, tm_sec=53, tm_wday=1, tm_yday=51, tm_isdst=0) 2024-02-20T02:14:57.103 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-02-20T02:14:59.062 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:14:59.063+0000 7f14bd1f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 23 scrub errors (OSD_SCRUB_ERRORS) 2024-02-20T02:14:59.062 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:14:59.064+0000 7f14bd1f6640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 23 pgs inconsistent (PG_DAMAGED) 2024-02-20T02:15:07.008 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:15:07.010+0000 7f14bd1f6640 -1 log_channel(cluster) log [ERR] : Health check update: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-02-20T02:15:07.009 INFO:tasks.ceph.mon.a.smithi125.stderr:2024-02-20T02:15:07.010+0000 7f14bd1f6640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 41 pgs inconsistent (PG_DAMAGED) 2024-02-20T02:15:17.104 DEBUG:teuthology.orchestra.run.smithi125:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-20T02:15:17.226 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.224+0000 7f92ed59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:15:17.227 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.229+0000 7f92ed59d640 1 Processor -- start 2024-02-20T02:15:17.227 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.229+0000 7f92ed59d640 1 -- start start 2024-02-20T02:15:17.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92ed59d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f92e8131c50 0x7f92e8132060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.229 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92ed59d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e8131890 con 0x7f92e806ae20 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92ed59d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e8072db0 con 0x7f92e8131c50 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e6ffd640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f92e8131c50 0x7f92e8132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92dffff640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f92e806ae20 0x7f92e806b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:57536/0 (socket says 172.21.15.125:57536) 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92dffff640 1 -- 172.21.15.125:0/3263583979 learned_addr learned my addr 172.21.15.125:0/3263583979 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e6ffd640 1 -- 172.21.15.125:0/3263583979 >> v1:172.21.15.125:6789/0 conn(0x7f92e806ae20 legacy=0x7f92e806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e6ffd640 1 -- 172.21.15.125:0/3263583979 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e809c700 con 0x7f92e8131c50 2024-02-20T02:15:17.230 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e67fc640 1 -- 172.21.15.125:0/3263583979 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058361230 0 0) 0x7f92e8131890 con 0x7f92e806ae20 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e6ffd640 1 --2- 172.21.15.125:0/3263583979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e8131c50 0x7f92e8132060 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f84d00452d839d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e67fc640 1 -- 172.21.15.125:0/3263583979 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92d8009ce0 con 0x7f92e8131c50 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e67fc640 1 -- 172.21.15.125:0/3263583979 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f92d8009e40 con 0x7f92e8131c50 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.230+0000 7f92e67fc640 1 -- 172.21.15.125:0/3263583979 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92d800e360 con 0x7f92e8131c50 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 -- 172.21.15.125:0/3263583979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e8131c50 msgr2=0x7f92e8132060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 --2- 172.21.15.125:0/3263583979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e8131c50 0x7f92e8132060 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.231 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 -- 172.21.15.125:0/3263583979 shutdown_connections 2024-02-20T02:15:17.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 --2- 172.21.15.125:0/3263583979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e8131c50 0x7f92e8132060 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 -- 172.21.15.125:0/3263583979 >> 172.21.15.125:0/3263583979 conn(0x7f92e809f890 msgr2=0x7f92e809fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:15:17.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 -- 172.21.15.125:0/3263583979 shutdown_connections 2024-02-20T02:15:17.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 1 -- 172.21.15.125:0/3263583979 wait complete. 2024-02-20T02:15:17.232 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.231+0000 7f92ed59d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:15:17.234 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.232+0000 7f92ed59d640 1 Processor -- start 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.232+0000 7f92ed59d640 1 -- start start 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92ed59d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92ed59d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e809ae60 con 0x7f92e806ae20 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92e6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92e6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35224/0 (socket says 172.21.15.125:35224) 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92e6ffd640 1 -- 172.21.15.125:0/4118875739 learned_addr learned my addr 172.21.15.125:0/4118875739 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:15:17.235 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92e6ffd640 1 -- 172.21.15.125:0/4118875739 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e809c160 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.233+0000 7f92e6ffd640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92d8014540 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f92d80146a0 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e809b700 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92d8014940 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e809ab50 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.234+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f92d8014b30 con 0x7f92e806ae20 2024-02-20T02:15:17.236 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.235+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e809bfc0 con 0x7f92e806ae20 2024-02-20T02:15:17.237 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.235+0000 7f92df7fe640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 0x7f92c803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.237 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.235+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f92d8019040 con 0x7f92e806ae20 2024-02-20T02:15:17.237 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.235+0000 7f92dffff640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 0x7f92c803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.237 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.236+0000 7f92dffff640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 0x7f92c803a3f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.237 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.240+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f92e809bfc0 con 0x7f92e806ae20 2024-02-20T02:15:17.411 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.414+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 --> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92e8074420 con 0x7f92c8037ee0 2024-02-20T02:15:17.415 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.417+0000 7f92df7fe640 1 -- 172.21.15.125:0/4118875739 <== mgr.4099 v2:172.21.15.204:6800/1051662976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107597 (crc 0 0 0) 0x7f92e8074420 con 0x7f92c8037ee0 2024-02-20T02:15:17.416 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-20T02:15:17.417 INFO:teuthology.orchestra.run.smithi125.stderr:dumped all 2024-02-20T02:15:17.418 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 msgr2=0x7f92c803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.418 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 0x7f92c803a3f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.418 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 msgr2=0x7f92e8099aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 shutdown_connections 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f92c8037ee0 0x7f92c803a3f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 --2- 172.21.15.125:0/4118875739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f92e806ae20 0x7f92e8099aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 >> 172.21.15.125:0/4118875739 conn(0x7f92e809f890 msgr2=0x7f92e81b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 shutdown_connections 2024-02-20T02:15:17.419 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.419+0000 7f92ed59d640 1 -- 172.21.15.125:0/4118875739 wait complete. 2024-02-20T02:15:17.427 INFO:teuthology.orchestra.run.smithi125.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-02-20T02:15:16.502910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":41,"num_shallow_scrub_errors":41,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":151448,"kb_used_data":10360,"kb_used_omap":0,"kb_used_meta":141056,"kb_avail":419278952,"statfs":{"total":429496729600,"available":429341646848,"internally_reserved":0,"allocated":10608640,"data_stored":4654222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":144441344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004016"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.185111+0000","last_change":"2024-02-20T02:14:55.865659+0000","last_active":"2024-02-20T02:15:00.185111+0000","last_peered":"2024-02-20T02:15:00.185111+0000","last_clean":"2024-02-20T02:15:00.185111+0000","last_became_active":"2024-02-20T02:14:44.170164+0000","last_became_peered":"2024-02-20T02:14:44.170164+0000","last_unstale":"2024-02-20T02:15:00.185111+0000","last_undegraded":"2024-02-20T02:15:00.185111+0000","last_fullsized":"2024-02-20T02:15:00.185111+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.865559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.865559+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.059857+0000","last_change":"2024-02-20T02:14:55.439881+0000","last_active":"2024-02-20T02:14:57.059857+0000","last_peered":"2024-02-20T02:14:57.059857+0000","last_clean":"2024-02-20T02:14:57.059857+0000","last_became_active":"2024-02-20T02:14:44.189022+0000","last_became_peered":"2024-02-20T02:14:44.189022+0000","last_unstale":"2024-02-20T02:14:57.059857+0000","last_undegraded":"2024-02-20T02:14:57.059857+0000","last_fullsized":"2024-02-20T02:14:57.059857+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.439779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.439779+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.204511+0000","last_change":"2024-02-20T02:14:56.328727+0000","last_active":"2024-02-20T02:15:00.204511+0000","last_peered":"2024-02-20T02:15:00.204511+0000","last_clean":"2024-02-20T02:15:00.204511+0000","last_became_active":"2024-02-20T02:14:44.173755+0000","last_became_peered":"2024-02-20T02:14:44.173755+0000","last_unstale":"2024-02-20T02:15:00.204511+0000","last_undegraded":"2024-02-20T02:15:00.204511+0000","last_fullsized":"2024-02-20T02:15:00.204511+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.328595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.328595+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.210586+0000","last_change":"2024-02-20T02:14:56.789212+0000","last_active":"2024-02-20T02:15:00.210586+0000","last_peered":"2024-02-20T02:15:00.210586+0000","last_clean":"2024-02-20T02:15:00.210586+0000","last_became_active":"2024-02-20T02:14:44.203048+0000","last_became_peered":"2024-02-20T02:14:44.203048+0000","last_unstale":"2024-02-20T02:15:00.210586+0000","last_undegraded":"2024-02-20T02:15:00.210586+0000","last_fullsized":"2024-02-20T02:15:00.210586+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.789053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.789053+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.200354+0000","last_change":"2024-02-20T02:14:55.868645+0000","last_active":"2024-02-20T02:15:00.200354+0000","last_peered":"2024-02-20T02:15:00.200354+0000","last_clean":"2024-02-20T02:15:00.200354+0000","last_became_active":"2024-02-20T02:14:44.201917+0000","last_became_peered":"2024-02-20T02:14:44.201917+0000","last_unstale":"2024-02-20T02:15:00.200354+0000","last_undegraded":"2024-02-20T02:15:00.200354+0000","last_fullsized":"2024-02-20T02:15:00.200354+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.868537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.868537+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.042518+0000","last_change":"2024-02-20T02:14:55.438448+0000","last_active":"2024-02-20T02:14:57.042518+0000","last_peered":"2024-02-20T02:14:57.042518+0000","last_clean":"2024-02-20T02:14:57.042518+0000","last_became_active":"2024-02-20T02:14:44.176812+0000","last_became_peered":"2024-02-20T02:14:44.176812+0000","last_unstale":"2024-02-20T02:14:57.042518+0000","last_undegraded":"2024-02-20T02:14:57.042518+0000","last_fullsized":"2024-02-20T02:14:57.042518+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.438342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.438342+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.066801+0000","last_change":"2024-02-20T02:14:55.444311+0000","last_active":"2024-02-20T02:14:57.066801+0000","last_peered":"2024-02-20T02:14:57.066801+0000","last_clean":"2024-02-20T02:14:57.066801+0000","last_became_active":"2024-02-20T02:14:44.173660+0000","last_became_peered":"2024-02-20T02:14:44.173660+0000","last_unstale":"2024-02-20T02:14:57.066801+0000","last_undegraded":"2024-02-20T02:14:57.066801+0000","last_fullsized":"2024-02-20T02:14:57.066801+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.444217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.444217+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.063703+0000","last_change":"2024-02-20T02:14:55.445088+0000","last_active":"2024-02-20T02:14:57.063703+0000","last_peered":"2024-02-20T02:14:57.063703+0000","last_clean":"2024-02-20T02:14:57.063703+0000","last_became_active":"2024-02-20T02:14:44.193652+0000","last_became_peered":"2024-02-20T02:14:44.193652+0000","last_unstale":"2024-02-20T02:14:57.063703+0000","last_undegraded":"2024-02-20T02:14:57.063703+0000","last_fullsized":"2024-02-20T02:14:57.063703+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.444985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.444985+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.225717+0000","last_change":"2024-02-20T02:14:56.787979+0000","last_active":"2024-02-20T02:15:00.225717+0000","last_peered":"2024-02-20T02:15:00.225717+0000","last_clean":"2024-02-20T02:15:00.225717+0000","last_became_active":"2024-02-20T02:14:44.199375+0000","last_became_peered":"2024-02-20T02:14:44.199375+0000","last_unstale":"2024-02-20T02:15:00.225717+0000","last_undegraded":"2024-02-20T02:15:00.225717+0000","last_fullsized":"2024-02-20T02:15:00.225717+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.787846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.787846+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.209972+0000","last_change":"2024-02-20T02:14:56.326103+0000","last_active":"2024-02-20T02:15:00.209972+0000","last_peered":"2024-02-20T02:15:00.209972+0000","last_clean":"2024-02-20T02:15:00.209972+0000","last_became_active":"2024-02-20T02:14:44.200473+0000","last_became_peered":"2024-02-20T02:14:44.200473+0000","last_unstale":"2024-02-20T02:15:00.209972+0000","last_undegraded":"2024-02-20T02:15:00.209972+0000","last_fullsized":"2024-02-20T02:15:00.209972+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.325969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.325969+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.211538+0000","last_change":"2024-02-20T02:14:56.786176+0000","last_active":"2024-02-20T02:15:00.211538+0000","last_peered":"2024-02-20T02:15:00.211538+0000","last_clean":"2024-02-20T02:15:00.211538+0000","last_became_active":"2024-02-20T02:14:44.190055+0000","last_became_peered":"2024-02-20T02:14:44.190055+0000","last_unstale":"2024-02-20T02:15:00.211538+0000","last_undegraded":"2024-02-20T02:15:00.211538+0000","last_fullsized":"2024-02-20T02:15:00.211538+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.786052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.786052+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.034899+0000","last_change":"2024-02-20T02:14:55.443816+0000","last_active":"2024-02-20T02:14:57.034899+0000","last_peered":"2024-02-20T02:14:57.034899+0000","last_clean":"2024-02-20T02:14:57.034899+0000","last_became_active":"2024-02-20T02:14:44.184534+0000","last_became_peered":"2024-02-20T02:14:44.184534+0000","last_unstale":"2024-02-20T02:14:57.034899+0000","last_undegraded":"2024-02-20T02:14:57.034899+0000","last_fullsized":"2024-02-20T02:14:57.034899+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.443716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.443716+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.205945+0000","last_change":"2024-02-20T02:14:56.783657+0000","last_active":"2024-02-20T02:15:00.205945+0000","last_peered":"2024-02-20T02:15:00.205945+0000","last_clean":"2024-02-20T02:15:00.205945+0000","last_became_active":"2024-02-20T02:14:44.192136+0000","last_became_peered":"2024-02-20T02:14:44.192136+0000","last_unstale":"2024-02-20T02:15:00.205945+0000","last_undegraded":"2024-02-20T02:15:00.205945+0000","last_fullsized":"2024-02-20T02:15:00.205945+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.783527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.783527+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.219033+0000","last_change":"2024-02-20T02:14:56.781254+0000","last_active":"2024-02-20T02:15:00.219033+0000","last_peered":"2024-02-20T02:15:00.219033+0000","last_clean":"2024-02-20T02:15:00.219033+0000","last_became_active":"2024-02-20T02:14:44.254178+0000","last_became_peered":"2024-02-20T02:14:44.254178+0000","last_unstale":"2024-02-20T02:15:00.219033+0000","last_undegraded":"2024-02-20T02:15:00.219033+0000","last_fullsized":"2024-02-20T02:15:00.219033+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.781114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.781114+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.061402+0000","last_change":"2024-02-20T02:14:55.442624+0000","last_active":"2024-02-20T02:14:57.061402+0000","last_peered":"2024-02-20T02:14:57.061402+0000","last_clean":"2024-02-20T02:14:57.061402+0000","last_became_active":"2024-02-20T02:14:44.250168+0000","last_became_peered":"2024-02-20T02:14:44.250168+0000","last_unstale":"2024-02-20T02:14:57.061402+0000","last_undegraded":"2024-02-20T02:14:57.061402+0000","last_fullsized":"2024-02-20T02:14:57.061402+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.442520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.442520+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.038919+0000","last_change":"2024-02-20T02:14:55.440970+0000","last_active":"2024-02-20T02:14:57.038919+0000","last_peered":"2024-02-20T02:14:57.038919+0000","last_clean":"2024-02-20T02:14:57.038919+0000","last_became_active":"2024-02-20T02:14:44.208652+0000","last_became_peered":"2024-02-20T02:14:44.208652+0000","last_unstale":"2024-02-20T02:14:57.038919+0000","last_undegraded":"2024-02-20T02:14:57.038919+0000","last_fullsized":"2024-02-20T02:14:57.038919+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.440859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.440859+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.053489+0000","last_change":"2024-02-20T02:14:55.441997+0000","last_active":"2024-02-20T02:14:57.053489+0000","last_peered":"2024-02-20T02:14:57.053489+0000","last_clean":"2024-02-20T02:14:57.053489+0000","last_became_active":"2024-02-20T02:14:44.197136+0000","last_became_peered":"2024-02-20T02:14:44.197136+0000","last_unstale":"2024-02-20T02:14:57.053489+0000","last_undegraded":"2024-02-20T02:14:57.053489+0000","last_fullsized":"2024-02-20T02:14:57.053489+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.441891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.441891+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.226586+0000","last_change":"2024-02-20T02:14:56.784249+0000","last_active":"2024-02-20T02:15:00.226586+0000","last_peered":"2024-02-20T02:15:00.226586+0000","last_clean":"2024-02-20T02:15:00.226586+0000","last_became_active":"2024-02-20T02:14:44.218175+0000","last_became_peered":"2024-02-20T02:14:44.218175+0000","last_unstale":"2024-02-20T02:15:00.226586+0000","last_undegraded":"2024-02-20T02:15:00.226586+0000","last_fullsized":"2024-02-20T02:15:00.226586+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.784101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.784101+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.196681+0000","last_change":"2024-02-20T02:14:56.323629+0000","last_active":"2024-02-20T02:15:00.196681+0000","last_peered":"2024-02-20T02:15:00.196681+0000","last_clean":"2024-02-20T02:15:00.196681+0000","last_became_active":"2024-02-20T02:14:44.175757+0000","last_became_peered":"2024-02-20T02:14:44.175757+0000","last_unstale":"2024-02-20T02:15:00.196681+0000","last_undegraded":"2024-02-20T02:15:00.196681+0000","last_fullsized":"2024-02-20T02:15:00.196681+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.323493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.323493+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.198610+0000","last_change":"2024-02-20T02:14:55.870561+0000","last_active":"2024-02-20T02:15:00.198610+0000","last_peered":"2024-02-20T02:15:00.198610+0000","last_clean":"2024-02-20T02:15:00.198610+0000","last_became_active":"2024-02-20T02:14:44.167479+0000","last_became_peered":"2024-02-20T02:14:44.167479+0000","last_unstale":"2024-02-20T02:15:00.198610+0000","last_undegraded":"2024-02-20T02:15:00.198610+0000","last_fullsized":"2024-02-20T02:15:00.198610+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.870458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.870458+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.205797+0000","last_change":"2024-02-20T02:14:55.869662+0000","last_active":"2024-02-20T02:15:00.205797+0000","last_peered":"2024-02-20T02:15:00.205797+0000","last_clean":"2024-02-20T02:15:00.205797+0000","last_became_active":"2024-02-20T02:14:44.180769+0000","last_became_peered":"2024-02-20T02:14:44.180769+0000","last_unstale":"2024-02-20T02:15:00.205797+0000","last_undegraded":"2024-02-20T02:15:00.205797+0000","last_fullsized":"2024-02-20T02:15:00.205797+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.869563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.869563+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.199549+0000","last_change":"2024-02-20T02:14:55.867652+0000","last_active":"2024-02-20T02:15:00.199549+0000","last_peered":"2024-02-20T02:15:00.199549+0000","last_clean":"2024-02-20T02:15:00.199549+0000","last_became_active":"2024-02-20T02:14:44.182801+0000","last_became_peered":"2024-02-20T02:14:44.182801+0000","last_unstale":"2024-02-20T02:15:00.199549+0000","last_undegraded":"2024-02-20T02:15:00.199549+0000","last_fullsized":"2024-02-20T02:15:00.199549+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.867546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.867546+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.213042+0000","last_change":"2024-02-20T02:14:56.327637+0000","last_active":"2024-02-20T02:15:00.213042+0000","last_peered":"2024-02-20T02:15:00.213042+0000","last_clean":"2024-02-20T02:15:00.213042+0000","last_became_active":"2024-02-20T02:14:44.186481+0000","last_became_peered":"2024-02-20T02:14:44.186481+0000","last_unstale":"2024-02-20T02:15:00.213042+0000","last_undegraded":"2024-02-20T02:15:00.213042+0000","last_fullsized":"2024-02-20T02:15:00.213042+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.327511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.327511+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.222130+0000","last_change":"2024-02-20T02:14:56.786700+0000","last_active":"2024-02-20T02:15:00.222130+0000","last_peered":"2024-02-20T02:15:00.222130+0000","last_clean":"2024-02-20T02:15:00.222130+0000","last_became_active":"2024-02-20T02:14:44.243754+0000","last_became_peered":"2024-02-20T02:14:44.243754+0000","last_unstale":"2024-02-20T02:15:00.222130+0000","last_undegraded":"2024-02-20T02:15:00.222130+0000","last_fullsized":"2024-02-20T02:15:00.222130+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.786573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.786573+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.228662+0000","last_change":"2024-02-20T02:14:56.781913+0000","last_active":"2024-02-20T02:15:00.228662+0000","last_peered":"2024-02-20T02:15:00.228662+0000","last_clean":"2024-02-20T02:15:00.228662+0000","last_became_active":"2024-02-20T02:14:44.194528+0000","last_became_peered":"2024-02-20T02:14:44.194528+0000","last_unstale":"2024-02-20T02:15:00.228662+0000","last_undegraded":"2024-02-20T02:15:00.228662+0000","last_fullsized":"2024-02-20T02:15:00.228662+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.781795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.781795+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.025280+0000","last_change":"2024-02-20T02:14:55.440542+0000","last_active":"2024-02-20T02:14:57.025280+0000","last_peered":"2024-02-20T02:14:57.025280+0000","last_clean":"2024-02-20T02:14:57.025280+0000","last_became_active":"2024-02-20T02:14:44.244704+0000","last_became_peered":"2024-02-20T02:14:44.244704+0000","last_unstale":"2024-02-20T02:14:57.025280+0000","last_undegraded":"2024-02-20T02:14:57.025280+0000","last_fullsized":"2024-02-20T02:14:57.025280+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.440446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.440446+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.204933+0000","last_change":"2024-02-20T02:14:56.776315+0000","last_active":"2024-02-20T02:15:00.204933+0000","last_peered":"2024-02-20T02:15:00.204933+0000","last_clean":"2024-02-20T02:15:00.204933+0000","last_became_active":"2024-02-20T02:14:44.234764+0000","last_became_peered":"2024-02-20T02:14:44.234764+0000","last_unstale":"2024-02-20T02:15:00.204933+0000","last_undegraded":"2024-02-20T02:15:00.204933+0000","last_fullsized":"2024-02-20T02:15:00.204933+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.776169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.776169+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.050285+0000","last_change":"2024-02-20T02:14:55.437369+0000","last_active":"2024-02-20T02:14:57.050285+0000","last_peered":"2024-02-20T02:14:57.050285+0000","last_clean":"2024-02-20T02:14:57.050285+0000","last_became_active":"2024-02-20T02:14:44.238057+0000","last_became_peered":"2024-02-20T02:14:44.238057+0000","last_unstale":"2024-02-20T02:14:57.050285+0000","last_undegraded":"2024-02-20T02:14:57.050285+0000","last_fullsized":"2024-02-20T02:14:57.050285+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.437263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.437263+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.070036+0000","last_change":"2024-02-20T02:14:55.436329+0000","last_active":"2024-02-20T02:14:57.070036+0000","last_peered":"2024-02-20T02:14:57.070036+0000","last_clean":"2024-02-20T02:14:57.070036+0000","last_became_active":"2024-02-20T02:14:44.182848+0000","last_became_peered":"2024-02-20T02:14:44.182848+0000","last_unstale":"2024-02-20T02:14:57.070036+0000","last_undegraded":"2024-02-20T02:14:57.070036+0000","last_fullsized":"2024-02-20T02:14:57.070036+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.436213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.436213+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.065683+0000","last_change":"2024-02-20T02:14:55.433071+0000","last_active":"2024-02-20T02:14:57.065683+0000","last_peered":"2024-02-20T02:14:57.065683+0000","last_clean":"2024-02-20T02:14:57.065683+0000","last_became_active":"2024-02-20T02:14:44.191242+0000","last_became_peered":"2024-02-20T02:14:44.191242+0000","last_unstale":"2024-02-20T02:14:57.065683+0000","last_undegraded":"2024-02-20T02:14:57.065683+0000","last_fullsized":"2024-02-20T02:14:57.065683+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.432969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.432969+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"238'7","reported_seq":47,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.195429+0000","last_change":"2024-02-20T02:14:55.871256+0000","last_active":"2024-02-20T02:15:00.195429+0000","last_peered":"2024-02-20T02:15:00.195429+0000","last_clean":"2024-02-20T02:15:00.195429+0000","last_became_active":"2024-02-20T02:14:44.172077+0000","last_became_peered":"2024-02-20T02:14:44.172077+0000","last_unstale":"2024-02-20T02:15:00.195429+0000","last_undegraded":"2024-02-20T02:15:00.195429+0000","last_fullsized":"2024-02-20T02:15:00.195429+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"238'7","last_scrub_stamp":"2024-02-20T02:14:55.871139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.871139+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":34,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.188693+0000","last_change":"2024-02-20T02:14:56.324339+0000","last_active":"2024-02-20T02:15:00.188693+0000","last_peered":"2024-02-20T02:15:00.188693+0000","last_clean":"2024-02-20T02:15:00.188693+0000","last_became_active":"2024-02-20T02:14:44.171906+0000","last_became_peered":"2024-02-20T02:14:44.171906+0000","last_unstale":"2024-02-20T02:15:00.188693+0000","last_undegraded":"2024-02-20T02:15:00.188693+0000","last_fullsized":"2024-02-20T02:15:00.188693+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.324206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.324206+0000","last_clean_scrub_stamp":"2024-02-20T02:14:43.010974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":620,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.201537+0000","last_change":"2024-02-20T02:14:55.866513+0000","last_active":"2024-02-20T02:15:00.201537+0000","last_peered":"2024-02-20T02:15:00.201537+0000","last_clean":"2024-02-20T02:15:00.201537+0000","last_became_active":"2024-02-20T02:09:44.202731+0000","last_became_peered":"2024-02-20T02:09:44.202731+0000","last_unstale":"2024-02-20T02:15:00.201537+0000","last_undegraded":"2024-02-20T02:15:00.201537+0000","last_fullsized":"2024-02-20T02:15:00.201537+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.866414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.866414+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":614,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.217997+0000","last_change":"2024-02-20T02:14:56.319162+0000","last_active":"2024-02-20T02:15:00.217997+0000","last_peered":"2024-02-20T02:15:00.217997+0000","last_clean":"2024-02-20T02:15:00.217997+0000","last_became_active":"2024-02-20T02:09:44.210459+0000","last_became_peered":"2024-02-20T02:09:44.210459+0000","last_unstale":"2024-02-20T02:15:00.217997+0000","last_undegraded":"2024-02-20T02:15:00.217997+0000","last_fullsized":"2024-02-20T02:15:00.217997+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.319006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.319006+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":626,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.269849+0000","last_change":"2024-02-20T02:14:56.772837+0000","last_active":"2024-02-20T02:15:00.269849+0000","last_peered":"2024-02-20T02:15:00.269849+0000","last_clean":"2024-02-20T02:15:00.269849+0000","last_became_active":"2024-02-20T02:09:44.204330+0000","last_became_peered":"2024-02-20T02:09:44.204330+0000","last_unstale":"2024-02-20T02:15:00.269849+0000","last_undegraded":"2024-02-20T02:15:00.269849+0000","last_fullsized":"2024-02-20T02:15:00.269849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.772588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.772588+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":614,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.203112+0000","last_change":"2024-02-20T02:14:56.325065+0000","last_active":"2024-02-20T02:15:00.203112+0000","last_peered":"2024-02-20T02:15:00.203112+0000","last_clean":"2024-02-20T02:15:00.203112+0000","last_became_active":"2024-02-20T02:09:44.192441+0000","last_became_peered":"2024-02-20T02:09:44.192441+0000","last_unstale":"2024-02-20T02:15:00.203112+0000","last_undegraded":"2024-02-20T02:15:00.203112+0000","last_fullsized":"2024-02-20T02:15:00.203112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:56.324896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.324896+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":620,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.040055+0000","last_change":"2024-02-20T02:14:55.432563+0000","last_active":"2024-02-20T02:14:57.040055+0000","last_peered":"2024-02-20T02:14:57.040055+0000","last_clean":"2024-02-20T02:14:57.040055+0000","last_became_active":"2024-02-20T02:09:44.197891+0000","last_became_peered":"2024-02-20T02:09:44.197891+0000","last_unstale":"2024-02-20T02:14:57.040055+0000","last_undegraded":"2024-02-20T02:14:57.040055+0000","last_fullsized":"2024-02-20T02:14:57.040055+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.432461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.432461+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":620,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.196925+0000","last_change":"2024-02-20T02:14:55.863654+0000","last_active":"2024-02-20T02:15:00.196925+0000","last_peered":"2024-02-20T02:15:00.196925+0000","last_clean":"2024-02-20T02:15:00.196925+0000","last_became_active":"2024-02-20T02:09:44.192408+0000","last_became_peered":"2024-02-20T02:09:44.192408+0000","last_unstale":"2024-02-20T02:15:00.196925+0000","last_undegraded":"2024-02-20T02:15:00.196925+0000","last_fullsized":"2024-02-20T02:15:00.196925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.863541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.863541+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":620,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:14:57.067474+0000","last_change":"2024-02-20T02:14:55.429865+0000","last_active":"2024-02-20T02:14:57.067474+0000","last_peered":"2024-02-20T02:14:57.067474+0000","last_clean":"2024-02-20T02:14:57.067474+0000","last_became_active":"2024-02-20T02:09:44.201110+0000","last_became_peered":"2024-02-20T02:09:44.201110+0000","last_unstale":"2024-02-20T02:14:57.067474+0000","last_undegraded":"2024-02-20T02:14:57.067474+0000","last_fullsized":"2024-02-20T02:14:57.067474+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.429739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.429739+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":620,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.212454+0000","last_change":"2024-02-20T02:14:55.859675+0000","last_active":"2024-02-20T02:15:00.212454+0000","last_peered":"2024-02-20T02:15:00.212454+0000","last_clean":"2024-02-20T02:15:00.212454+0000","last_became_active":"2024-02-20T02:09:44.210014+0000","last_became_peered":"2024-02-20T02:09:44.210014+0000","last_unstale":"2024-02-20T02:15:00.212454+0000","last_undegraded":"2024-02-20T02:15:00.212454+0000","last_fullsized":"2024-02-20T02:15:00.212454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-20T02:14:55.859485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:55.859485+0000","last_clean_scrub_stamp":"2024-02-20T02:09:43.147664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":762,"reported_epoch":246,"state":"active+clean+inconsistent","last_fresh":"2024-02-20T02:15:00.216290+0000","last_change":"2024-02-20T02:14:56.790706+0000","last_active":"2024-02-20T02:15:00.216290+0000","last_peered":"2024-02-20T02:15:00.216290+0000","last_clean":"2024-02-20T02:15:00.216290+0000","last_became_active":"2024-02-20T02:09:40.187203+0000","last_became_peered":"2024-02-20T02:09:40.187203+0000","last_unstale":"2024-02-20T02:15:00.216290+0000","last_undegraded":"2024-02-20T02:15:00.216290+0000","last_fullsized":"2024-02-20T02:15:00.216290+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"11'97","last_scrub_stamp":"2024-02-20T02:14:56.790556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-20T02:14:56.790556+0000","last_clean_scrub_stamp":"2024-02-20T02:09:39.134569+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":32,"num_shallow_scrub_errors":32,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":8,"num_shallow_scrub_errors":8,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":42332,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815268,"statfs":{"total":107374182400,"available":107330834432,"internally_reserved":0,"allocated":2969600,"data_stored":1474321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949673294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":37568,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":35264,"kb_avail":104820032,"statfs":{"total":107374182400,"available":107335712768,"internally_reserved":0,"allocated":2351104,"data_stored":863222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36110336},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":41724,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":104815876,"statfs":{"total":107374182400,"available":107331457024,"internally_reserved":0,"allocated":2347008,"data_stored":863094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":29824,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104827776,"statfs":{"total":107374182400,"available":107343642624,"internally_reserved":0,"allocated":2940928,"data_stored":1453585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-02-20T02:15:17.431 DEBUG:teuthology.orchestra.run.smithi125:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-02-20T02:15:17.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.533+0000 7f059bb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:15:17.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.539+0000 7f059bb9d640 1 Processor -- start 2024-02-20T02:15:17.537 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f059bb9d640 1 -- start start 2024-02-20T02:15:17.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f059bb9d640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f059409bc40 0x7f0594099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f059bb9d640 1 -- --> v1:172.21.15.125:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0594099a70 con 0x7f059409a3e0 2024-02-20T02:15:17.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f059bb9d640 1 -- --> v2:172.21.15.125:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059409ae60 con 0x7f059409bc40 2024-02-20T02:15:17.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f0599939640 1 --2- >> v2:172.21.15.125:3300/0 conn(0x7f059409bc40 0x7f0594099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.539 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.540+0000 7f0599138640 1 --1- >> v1:172.21.15.125:6789/0 conn(0x7f059409a3e0 0x7f0594131890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.125:6789/0 says I am v1:172.21.15.125:57552/0 (socket says 172.21.15.125:57552) 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0599138640 1 -- 172.21.15.125:0/1627752416 learned_addr learned my addr 172.21.15.125:0/1627752416 (peer_addr_for_me v1:172.21.15.125:0/0) 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0599939640 1 -- 172.21.15.125:0/1627752416 >> v1:172.21.15.125:6789/0 conn(0x7f059409a3e0 legacy=0x7f0594131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0599939640 1 -- 172.21.15.125:0/1627752416 --> v2:172.21.15.125:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059409b4d0 con 0x7f059409bc40 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0598937640 1 -- 172.21.15.125:0/1627752416 <== mon.0 v1:172.21.15.125:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399843285 0 0) 0x7f0594099a70 con 0x7f059409a3e0 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0599939640 1 --2- 172.21.15.125:0/1627752416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409bc40 0x7f0594099ea0 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33215e1b5acdfe32 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0598937640 1 -- 172.21.15.125:0/1627752416 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0584009ce0 con 0x7f059409bc40 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0598937640 1 -- 172.21.15.125:0/1627752416 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0584009e40 con 0x7f059409bc40 2024-02-20T02:15:17.540 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.541+0000 7f0598937640 1 -- 172.21.15.125:0/1627752416 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f058400e360 con 0x7f059409bc40 2024-02-20T02:15:17.543 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 -- 172.21.15.125:0/1627752416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409bc40 msgr2=0x7f0594099ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 --2- 172.21.15.125:0/1627752416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409bc40 0x7f0594099ea0 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 -- 172.21.15.125:0/1627752416 shutdown_connections 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 --2- 172.21.15.125:0/1627752416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409bc40 0x7f0594099ea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 -- 172.21.15.125:0/1627752416 >> 172.21.15.125:0/1627752416 conn(0x7f059409fae0 msgr2=0x7f059409fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 -- 172.21.15.125:0/1627752416 shutdown_connections 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 1 -- 172.21.15.125:0/1627752416 wait complete. 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.542+0000 7f059bb9d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-02-20T02:15:17.544 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.543+0000 7f059bb9d640 1 Processor -- start 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.543+0000 7f059bb9d640 1 -- start start 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f059bb9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f059bb9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059411c3d0 con 0x7f059409a3e0 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f0599939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f0599939640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:35248/0 (socket says 172.21.15.125:35248) 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f0599939640 1 -- 172.21.15.125:0/2242612749 learned_addr learned my addr 172.21.15.125:0/2242612749 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f0599939640 1 -- 172.21.15.125:0/2242612749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059411ef20 con 0x7f059409a3e0 2024-02-20T02:15:17.545 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f0599939640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0584014540 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f05840146a0 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0584014940 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059412f160 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.544+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059412ee50 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.545+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 5) v1 ==== 45208+0+0 (crc 0 0 0) 0x7f0584014af0 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.545+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0594062a30 con 0x7f059409a3e0 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.545+0000 7f058a7fc640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 0x7f057803a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-20T02:15:17.546 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.545+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 3035+0+0 (crc 0 0 0) 0x7f0584019040 con 0x7f059409a3e0 2024-02-20T02:15:17.547 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.546+0000 7f0599138640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 0x7f057803a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-20T02:15:17.547 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.546+0000 7f0599138640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 0x7f057803a3f0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T02:15:17.547 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.550+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (crc 0 0 0) 0x7f0584018190 con 0x7f059409a3e0 2024-02-20T02:15:17.720 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.722+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0594068410 con 0x7f059409a3e0 2024-02-20T02:15:17.723 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.726+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (crc 0 0 0) 0x7f058400fb30 con 0x7f059409a3e0 2024-02-20T02:15:17.724 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.726+0000 7f058a7fc640 1 -- 172.21.15.125:0/2242612749 <== mon.0 v2:172.21.15.125:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (crc 0 0 0) 0x7f0584018190 con 0x7f059409a3e0 2024-02-20T02:15:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 msgr2=0x7f057803a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 0x7f057803a3f0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 msgr2=0x7f059411af40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T02:15:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.727 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 shutdown_connections 2024-02-20T02:15:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.204:6800/1051662976,v1:172.21.15.204:6801/1051662976] conn(0x7f0578037ee0 0x7f057803a3f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 --2- 172.21.15.125:0/2242612749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f059409a3e0 0x7f059411af40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-20T02:15:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 >> 172.21.15.125:0/2242612749 conn(0x7f059409fae0 msgr2=0x7f05941b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T02:15:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 shutdown_connections 2024-02-20T02:15:17.728 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-20T02:15:17.729+0000 7f059bb9d640 1 -- 172.21.15.125:0/2242612749 wait complete. 2024-02-20T02:15:17.738 INFO:teuthology.misc:Shutting down mds daemons... 2024-02-20T02:15:17.739 INFO:teuthology.misc:Shutting down osd daemons... 2024-02-20T02:15:17.739 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-02-20T02:15:17.739 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:17.834 INFO:tasks.ceph.osd.0:Stopped 2024-02-20T02:15:17.835 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-02-20T02:15:17.835 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:17.926 INFO:tasks.ceph.osd.1:Stopped 2024-02-20T02:15:17.926 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-02-20T02:15:17.926 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:20.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-02-20T02:15:24.029 INFO:tasks.ceph.osd.2:Stopped 2024-02-20T02:15:24.029 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-02-20T02:15:24.029 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:24.127 INFO:tasks.ceph.osd.3:Stopped 2024-02-20T02:15:24.127 INFO:teuthology.misc:Shutting down mgr daemons... 2024-02-20T02:15:24.127 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-02-20T02:15:24.128 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:24.156 INFO:tasks.ceph.mgr.x:Stopped 2024-02-20T02:15:24.156 INFO:teuthology.misc:Shutting down mon daemons... 2024-02-20T02:15:24.157 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-02-20T02:15:24.157 INFO:teuthology.orchestra.run:waiting for 300 2024-02-20T02:15:24.170 INFO:tasks.ceph.mon.a:Stopped 2024-02-20T02:15:24.171 INFO:tasks.ceph:Checking cluster log for badness... 2024-02-20T02:15:24.171 DEBUG:teuthology.orchestra.run.smithi125:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-20T02:15:24.244 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:15:24.245 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-02-20T02:15:24.459 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:15:24.460 DEBUG:teuthology.orchestra.run.smithi125:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-02-20T02:15:24.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:15:24.534 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-02-20T02:15:24.942 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:15:24.942 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-02-20T02:15:25.017 INFO:tasks.ceph:Archiving mon data... 2024-02-20T02:15:25.018 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/data/mon.a.tgz 2024-02-20T02:15:25.018 DEBUG:teuthology.orchestra.run.smithi125:> mktemp 2024-02-20T02:15:25.037 INFO:teuthology.orchestra.run.smithi125.stdout:/tmp/tmp.trKSFltKVn 2024-02-20T02:15:25.038 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.trKSFltKVn 2024-02-20T02:15:25.196 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 0666 /tmp/tmp.trKSFltKVn 2024-02-20T02:15:25.281 DEBUG:teuthology.orchestra.remote:smithi125:/tmp/tmp.trKSFltKVn is 392KB 2024-02-20T02:15:25.379 DEBUG:teuthology.orchestra.run.smithi125:> rm -fr /tmp/tmp.trKSFltKVn 2024-02-20T02:15:25.394 INFO:tasks.ceph:Cleaning ceph cluster... 2024-02-20T02:15:25.394 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-20T02:15:25.436 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-20T02:15:25.515 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-02-20T02:15:25.536 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-02-20T02:15:25.536 INFO:tasks.ceph:Archiving crash dumps... 2024-02-20T02:15:25.538 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/crash to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi125/crash 2024-02-20T02:15:25.538 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-20T02:15:25.582 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi204/crash 2024-02-20T02:15:25.583 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-20T02:15:25.628 INFO:tasks.ceph:Compressing logs... 2024-02-20T02:15:25.628 DEBUG:teuthology.orchestra.run.smithi125:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-20T02:15:25.631 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-20T02:15:25.665 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30233.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30233.log: /var/log/ceph/ceph-client.admin.30313.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30233.log.gz 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30193.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30273.log 2024-02-20T02:15:25.666 INFO:teuthology.orchestra.run.smithi204.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30193.log.gz 2024-02-20T02:15:25.667 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-02-20T02:15:25.667 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30273.log: /var/log/ceph/ceph-osd.2.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30273.log.gz 2024-02-20T02:15:25.675 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-02-20T02:15:25.675 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34616.log 2024-02-20T02:15:25.675 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.34616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33911.log 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34576.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.34616.log.gz 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32204.log 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.33911.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32203.log 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33911.log.gz 2024-02-20T02:15:25.676 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34364.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.32204.log.gz 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33950.log 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32203.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.32203.log.gz 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.33950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32397.log 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34153.log 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32397.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-02-20T02:15:25.677 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32397.log.gz 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34492.log 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34153.log: /var/log/ceph/ceph-client.admin.31893.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.34153.log.gz 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34492.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30623.log 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.34492.log.gz 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28292.log 2024-02-20T02:15:25.678 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.30623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31767.log 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30623.log.gz 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.tmp-client.admin.28292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31711.log 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr: 68.3% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28292.log.gz 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34408.log 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.31767.log: /var/log/ceph/ceph-client.admin.31711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30702.log 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.31767.log.gz 2024-02-20T02:15:25.679 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34408.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31711.log.gz 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.34408.log.gz 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34111.log 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.30702.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34321.log 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30702.log.gz 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34195.log 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.34111.log.gz 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33892.log 2024-02-20T02:15:25.680 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34195.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.34321.log.gz 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30859.log 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34195.log.gz 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.33892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34534.log 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.30859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30887.log 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30859.log.gz 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32313.log 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.33892.log.gz 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.30887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32202.log 2024-02-20T02:15:25.681 INFO:teuthology.orchestra.run.smithi125.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.34534.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.33950.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30887.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34027.log 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.32313.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31935.log 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32202.log: /var/log/ceph/ceph-client.admin.34027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33858.log 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32202.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.31935.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.34027.log.gz 2024-02-20T02:15:25.682 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32035.log 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.31935.log.gz 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.33858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31851.log 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32035.log.gz 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33858.log.gz 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32037.log 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.31851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-02-20T02:15:25.683 INFO:teuthology.orchestra.run.smithi125.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32037.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31851.log.gz 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.33086.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.32037.log.gz 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31809.log 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32036.log 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.31809.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32036.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31809.log.gz 2024-02-20T02:15:25.684 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32617.log 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32036.log.gz 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32355.log 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34450.log 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34279.log 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.32355.log.gz 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34450.log: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.34450.log.gz 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32034.log 2024-02-20T02:15:25.685 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.34279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32205.log 2024-02-20T02:15:25.686 INFO:teuthology.orchestra.run.smithi125.stderr: 86.7%/var/log/ceph/ceph-client.admin.32034.log: -- replaced with /var/log/ceph/ceph-client.admin.34279.log.gz 2024-02-20T02:15:25.686 INFO:teuthology.orchestra.run.smithi125.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.32034.log.gz 2024-02-20T02:15:25.686 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/ceph-client.admin.32205.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.32205.log.gz 2024-02-20T02:15:25.686 INFO:teuthology.orchestra.run.smithi125.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.32617.log.gz 2024-02-20T02:15:25.686 INFO:teuthology.orchestra.run.smithi125.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-02-20T02:15:25.700 INFO:teuthology.orchestra.run.smithi204.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-02-20T02:15:25.921 INFO:teuthology.orchestra.run.smithi125.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-02-20T02:15:25.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-02-20T02:15:26.833 INFO:teuthology.orchestra.run.smithi204.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-02-20T02:15:26.839 INFO:teuthology.orchestra.run.smithi125.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-02-20T02:15:26.865 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-02-20T02:15:26.867 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-02-20T02:15:26.867 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m1.222s 2024-02-20T02:15:26.867 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m2.317s 2024-02-20T02:15:26.867 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.130s 2024-02-20T02:15:27.012 INFO:teuthology.orchestra.run.smithi125.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-02-20T02:15:27.014 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-20T02:15:27.014 INFO:teuthology.orchestra.run.smithi125.stderr:real 0m1.372s 2024-02-20T02:15:27.014 INFO:teuthology.orchestra.run.smithi125.stderr:user 0m2.734s 2024-02-20T02:15:27.014 INFO:teuthology.orchestra.run.smithi125.stderr:sys 0m0.146s 2024-02-20T02:15:27.014 INFO:tasks.ceph:Archiving logs... 2024-02-20T02:15:27.015 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/log/ceph to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi125/log 2024-02-20T02:15:27.016 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-20T02:15:27.561 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi204/log 2024-02-20T02:15:27.562 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-20T02:15:27.992 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-20T02:15:28.002 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-20T02:15:28.002 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-20T02:15:28.005 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-20T02:15:28.097 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-20T02:15:28.106 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-20T02:15:28.106 DEBUG:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T02:15:28.109 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T02:15:28.121 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2024-02-20T02:15:28.124 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-02-20T02:15:28.125 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T02:15:28.125 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2024-02-20T02:15:28.125 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +2804us[+2807us] +/- 83ms 2024-02-20T02:15:28.125 INFO:teuthology.orchestra.run.smithi125.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -492us[ -490us] +/- 71ms 2024-02-20T02:15:28.125 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -1982us[-1980us] +/- 76ms 2024-02-20T02:15:28.126 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:15:28.130 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T02:15:28.130 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-02-20T02:15:28.131 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +2816us[+2833us] +/- 83ms 2024-02-20T02:15:28.131 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -479us[ -463us] +/- 71ms 2024-02-20T02:15:28.131 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -1969us[-1953us] +/- 76ms 2024-02-20T02:15:28.131 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T02:15:28.131 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-20T02:15:28.141 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-20T02:15:28.142 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-20T02:15:28.186 DEBUG:teuthology.orchestra.run.smithi125:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-20T02:15:28.251 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-20T02:15:28.317 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T02:15:28.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:15:28.347 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T02:15:28.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:15:28.377 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-20T02:15:28.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-20T02:15:28.410 INFO:teuthology.task.internal:Duration was 866.506472 seconds 2024-02-20T02:15:28.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-20T02:15:28.460 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-20T02:15:28.461 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-20T02:15:28.463 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-20T02:15:28.504 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T02:15:28.508 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T02:15:28.640 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-20T02:15:28.640 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi125.front.sepia.ceph.com 2024-02-20T02:15:28.641 DEBUG:teuthology.orchestra.run.smithi125:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-20T02:15:28.700 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-02-20T02:15:28.701 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-20T02:15:28.725 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-20T02:15:28.725 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-20T02:15:28.743 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-20T02:15:28.794 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-20T02:15:28.794 DEBUG:teuthology.orchestra.run.smithi125:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-20T02:15:28.812 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-20T02:15:28.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-20T02:15:28.939 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-20T02:15:28.939 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-20T02:15:28.967 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-20T02:15:28.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-20T02:15:29.004 DEBUG:teuthology.orchestra.run.smithi125:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-20T02:15:29.009 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-20T02:15:29.032 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = core 2024-02-20T02:15:29.060 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-02-20T02:15:29.097 DEBUG:teuthology.orchestra.run.smithi125:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-20T02:15:29.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:15:29.111 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-20T02:15:29.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:15:29.152 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-20T02:15:29.162 INFO:teuthology.task.internal:Transferring archived files... 2024-02-20T02:15:29.163 DEBUG:teuthology.misc:Transferring archived files from smithi125:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi125 2024-02-20T02:15:29.163 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-20T02:15:29.199 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238/remote/smithi204 2024-02-20T02:15:29.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-20T02:15:29.238 INFO:teuthology.task.internal:Removing archive directory... 2024-02-20T02:15:29.239 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-20T02:15:29.241 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-20T02:15:29.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-20T02:15:29.297 INFO:teuthology.task.internal:Not uploading archives. 2024-02-20T02:15:29.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-20T02:15:29.319 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-20T02:15:29.320 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-20T02:15:29.322 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-20T02:15:29.334 INFO:teuthology.orchestra.run.smithi125.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 02:15 /home/ubuntu/cephtest 2024-02-20T02:15:29.334 INFO:teuthology.orchestra.run.smithi125.stdout: 532017 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 02:09 /home/ubuntu/cephtest/mnt.0 2024-02-20T02:15:29.334 INFO:teuthology.orchestra.run.smithi125.stdout: 532018 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 02:10 /home/ubuntu/cephtest/mnt.0/client.0 2024-02-20T02:15:29.334 INFO:teuthology.orchestra.run.smithi125.stdout: 669416 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 02:10 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-20T02:15:29.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T02:15:29.335 INFO:teuthology.orchestra.run.smithi125.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-20T02:15:29.336 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi125 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-20T02:15:29.344 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-20T02:15:29.354 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-20T02:15:29.387 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi125 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-20T02:15:29.387 INFO:teuthology.nuke:Checking targets against current locks 2024-02-20T02:15:29.394 INFO:teuthology.orchestra.run.smithi204.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 02:15 /home/ubuntu/cephtest 2024-02-20T02:15:29.412 DEBUG:teuthology.nuke:shortname: smithi125 2024-02-20T02:15:29.413 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T02:15:29.436 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 01:51:44.834395', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO3Qp7DETLHvlRbXOx2wpU4DdRiTNXdGoP1rxNFm4cBMOomqjvMx+/OclkALynycfzFVlOMbvPgxUrIL4pI3mi8='} 2024-02-20T02:15:29.443 DEBUG:teuthology.nuke:shortname: smithi204 2024-02-20T02:15:29.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T02:15:29.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-02-20_01:50:29-crimson-rados-wip-sjust-crimson-testing-2024-02-19-distro-default-smithi/7568238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 01:51:44.835825', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Q8rcGTT41i9niKJxNu8yqnottToCTCJw8aBW7rEmy1tCZ0L0RzXTjJ4/8mOQVVvrgN1ksvraVhD+S5Pp337M='} 2024-02-20T02:15:29.511 INFO:teuthology.orchestra.console.smithi204:Power off 2024-02-20T02:15:29.512 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-20T02:15:29.527 INFO:teuthology.orchestra.console.smithi125:Power off 2024-02-20T02:15:29.527 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-20T02:15:29.536 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-02-20T02:15:29.536 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:29.550 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2024-02-20T02:15:29.550 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:29.560 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-02-20T02:15:29.576 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-02-20T02:15:33.561 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:33.576 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:33.672 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-02-20T02:15:33.686 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-02-20T02:15:37.675 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:37.687 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T02:15:37.785 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-02-20T02:15:37.797 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-02-20T02:15:37.887 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-02-20T02:15:37.898 INFO:teuthology.orchestra.console.smithi125:Power off completed 2024-02-20T02:15:38.004 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 866.5064723491669 failure_reason: 'Command failed (workunit test rados/test.sh) on smithi125 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663dfa8925d03992983e2ae809f7a5fc9835cd6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson''' flavor: crimson owner: scheduled_sjust@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bd50b43740404b768caa79b94e0d0d81 status: fail success: false 2024-02-20T02:15:38.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-20T02:15:38.096 INFO:teuthology.run:FAIL